Posts Tagged '3D'



Gyroscope adventure

In case you can’t fully understand yet what Adobe means by focusing on (desktop) gaming and video and not general purpose apps, allow me to demonstrate. Suppose you are making mobile app and for whatever reason you want to track device orientation. How hard can that be? Check it out:

Apple way

If you’re native iOS app developer, you are dealing with ecosystem that does support general purpose apps. You have simple CMDeviceMotion class that performs sensor data fusion, filtering, bias removal, etc, and gives you orientation data in any form you want (roll + pitch + yaw, or matrix, or quaternion). Case closed. The same class conveniently splits accelerometer data into gravity and user applied acceleration, btw. In a word, it’s perfect.

Adobe way

If you’re AIR app developer, prepare yourself – you’re going on the adventure. You can start by looking at flash.sensors package – for some mysterious reason, it has accelerometer API for desktop, but no gyroscope API for mobile. Why? Because Adobe does not feel obliged to include every piece of code you might think of in their runtime. But don’t give up your hope yet – Adobe has the solution for this kind of problems – AIR native extensions. And, it turns out that Adobe made one for you to access gyroscope data. Despite of being abandoned in 2011, apparently just before Adobe’s focus shift kicked in, this ANE still works with current AIR 3.x.

Okay, this is the part where you’re happy that you don’t have to write ANE yourself, and it seems that getting your hands on device orientation data is now only a matter of few additional build settings. Well…

This extension API, designed to be Accelerometer class twin, has two major problems in the way. Number one is that it only exposes raw gyroscope data, i.e. rotation rate. Since this is not something that flash developers do every day, you probably have no idea how to go from radians per second to orientation quaternion, or transformation matrix. So to use this ANE, you will have to learn some new tricks, such as finding approximate differential equation solutions, Taylor expansion of matrix functions, series summation and evaluating indeterminate forms. If you are still with me, good guy Oliver J. Woodman will guide you through this hell (pages 21 to 23).

Ok, now you’re done reading that and ready for action, right? Not so fast – there is problem number two. Let me just quote this comment from ANE source code:

// The singleton ExtensionContext object listens for StatusEvent events that
// the native implementation dispatches. These events contain the device’s
// gyroscope x,y,z data.
//
// However, each Gyroscope instance has its own interval timer. When the timer
// expires, the Gyroscope instance dispatches a GyroscopeEvent that contains
// the current x,y,z data.

Thank you, Adobe engineer, for this brilliant idea. Because there just can’t be enough lag or noise in my data. Sigh… so what do you do about this? You bypass ANE AS3 wrapper, and access its internal methods over low level interface. Of course, this means that you have to study both wrapper and native code beforehand :(

And here, my friend, we come to the end of our wonderful journey. I will go back to work on my AIR app, and you will maybe go and write your own ANE, with blackjack and hookers. Or you may choose to continue where I left off – maybe add Kalman filter or something. And don’t forget to write back, if you do :)

WTF is Transformation Matrix?

A transformation matrix transforms a vector of point coordinates in given frame of reference into coordinates in another one (e.g. from child to parent in our beloved display lists or scene graphs) – Captain Obvious.

Many people think of transformation matrices as impenetrable black boxes filled with random numbers that are impossible to make sense of while fiddling with your application. So this 1st post of the year is my attempt to help those people, by providing as short and simple explanation for those numbers as possible.

P.S. Since matrices transform vectors, you need to know a thing or two about vectors 1st. Continue reading ‘WTF is Transformation Matrix?’

Grab my balls

So, I have started cross-engine 3D primitives project at github. Just like as3dmod, it plugs seamlessly into your favorite engine to help you out whenever you feel limited by it. Initial commit supports Alternativa3D 8.12, Away3D 4 Alpha, Minko 1 and contains two spheric primitives: UniformSphere and Globe.

UniformSphere is a mesh built from the spiral of vertices uniformly distributed across the sphere. You have two options here: Rakhmanov formula (default, the spiral joins sphere poles) or Bauer formula (the spiral does not contain poles). This primitive was ported from my 3D metaballs code.

Globe is a mesh that minimizes texture distortions for classic plate carré projection. It does so by placing equilateral triangles along the equator and gradually compressing triangles towards each pole (where high triangle density is actually good property in terms of texture distortions). This primitive was ported from my globe component for FP9, which is now also hosted at github.

I plan to add few more primitives over time, both useful and not really, and maybe more engine proxies if there will be any interest.

Still messing with qtrack

Ok, boring things first. I uploaded another qtrack version with some minor fixes and improvements, and also added broomstick example there. Nothing groundbreaking so far.

Now, to fun things. I tried to make non-square marker qtrack mod based on this method. You can see in this video how far I could push it:

but, overall, this was a failure. Unlike that guy, I had only one shape to detect, so at 1st I thought I could use max ICF value between blob and marker contours (ideally 1) and compare it to confidence threshold. However, this picked way too much false positives. So in 2nd iteration, that you see in the video, I had to calculate max ICF values for few hundred blobs 1st, and then pick the blob with max of those values. This worked somewhat better, but in the long run, it is still unreliable. Here is a distribution of max max ICF values thresholded at 0.7:

As you see, even shape as simple as flash logo produces really poor values most of the time. Another equally significant problem is that BitmapData’s threshold() method produces poor contours when threshold is not optimal: over-illuminated patches of the marker punch random “holes” in its binarized version and contaminate detected contour. To sum up, you will have to stick with quad markers for some time, sorry.

Meet qtrack

It stands for “quadrilateral tracker”, little pet project I have been sporadically working on since october. I should have given it more love, but I didn’t want this post to drag into next year, so here goes “version 0.9″:

After December 7th, qtrack is the only free, as in “free beer”, flash 3D AR tracker. Curious why? There are many reasons for that. For one, most of its code is already available for free elsewhere on the web. But another, more important reason is that I just don’t see enough flash AR around, and setting this thing free is my modest contribution to spark some more creativity in the field :) I mean, when was the last time you actually saw flash AR game, huh?..

Any way, back to qtrack – it works similarly to famous FLARToolKit, except it is not that advanced and currently implements only single instance of single quad marker tracking. On the bright side, it is less restrictive regarding marker shape – all these markers could be tracked with qtrack, for example:

So, this is something you can try today. Zip file comes with two simple examples for Away Lite and Alternativa engines, and soon short video tutorial will follow. In January I will try to further improve qtrack in the areas where it currently have problems, and release stable version 1.0.

Credits

As I said, most of this thing is based on free code snippets published or inspired by various people. This includes blob detection (Tomek), homography (Zeh), marching squares (Sakri), Otsu thresholding (Andrew), automatic thresholding (some unknown guy), brilliant camera activity trick (Deepanjan). Even my complex numbers experiment found its way into this – I used it for pose estimation. So big thanks goes to all these people who have posted right things at right time for me to come across – without you qtrack would never happen!


Old stuff

July 2014
M T W T F S S
« May    
 123456
78910111213
14151617181920
21222324252627
28293031  

Oh, btw…


Follow

Get every new post delivered to your Inbox.