A transformation matrix transforms a vector of point coordinates in given frame of reference into coordinates in another one (e.g. from child to parent in our beloved display lists or scene graphs) – Captain Obvious.

Many people think of transformation matrices as impenetrable black boxes filled with random numbers that are impossible to make sense of while fiddling with your application. So this 1st post of the year is my attempt to help those people, by providing as short and simple explanation for those numbers as possible.

P.S. Since matrices transform vectors, you need to know a thing or two about vectors 1st.

UniformSphere is a mesh built from the spiral of vertices uniformly distributed across the sphere. You have two options here: Rakhmanov formula (default, the spiral joins sphere poles) or Bauer formula (the spiral does not contain poles). This primitive was ported from my 3D metaballs code.

Globe is a mesh that minimizes texture distortions for classic plate carré projection. It does so by placing equilateral triangles along the equator and gradually compressing triangles towards each pole (where high triangle density is actually good property in terms of texture distortions). This primitive was ported from my globe component for FP9, which is now also hosted at github.

I plan to add few more primitives over time, both useful and not really, and maybe more engine proxies if there will be any interest.

My entry was heavily influenced by awesome PSTW Action RPG, which I really wish all RPGs were like.

Now, to fun things. I tried to make non-square marker qtrack mod based on this method. You can see in this video how far I could push it:

but, overall, this was a failure. Unlike that guy, I had only one shape to detect, so at 1st I thought I could use max ICF value between blob and marker contours (ideally 1) and compare it to confidence threshold. However, this picked way too much false positives. So in 2nd iteration, that you see in the video, I had to calculate max ICF values for few hundred blobs 1st, and then pick the blob with max of those values. This worked somewhat better, but in the long run, it is still unreliable. Here is a distribution of max max ICF values thresholded at 0.7:

As you see, even shape as simple as flash logo produces really poor values most of the time. Another equally significant problem is that BitmapData’s threshold() method produces poor contours when threshold is not optimal: over-illuminated patches of the marker punch random ”holes” in its binarized version and contaminate detected contour. To sum up, you will have to stick with quad markers for some time, sorry.

On a related note, back in December I promised to make Alternativa3D AR video tutorial for russian xmas, that would include qtrack and other trackers available through FLARManager. I did not finish that video in time, so for this post I uploaded qtrack part only (and chances are that FLARManager part will be only available in text version of tutorial). The video is in russian, but you can try english subtitles. Enjoy:

If you have been searching for this stuff before ending up here, you could notice that cool guys never do color matching in RGB, but instead go with HSV or similar hue-based color space. The idea here is, probably, that when you pull all color information into one coordinate, you can forget about other two. Unfortunately you can’t :( It’s not to say that there is some kind of a problem here, not at all. With tools like pixel bender on your belt, you just write a shader that converts to HSV and applies thresholds in one breath, and voila – you get your blobs to track. But, for some reason, this kind of blind belief in hue dogma just doesn’t sit well with me.

So, as the saying goes, ”know your enemy”. Let us first look at hue color wheel for a moment (on your left). Immediately we note one special thing about hue – it is measured in degrees. Which means it is an angle. And since it is an angle, there has to be corresponding axis in 3D color space, let’s see… Ha! In both HSL and HSV spaces this axis spans colors that go from black to white.

What a coincidence, but RGB space also has same axis. It is RGB cube diagonal that goes from R=0 G=0 B=0 to R=255 G=255 B=255. So what does RGB cube look like, when you look at it along this direction? Well, here comes another handy picture from wikimedia (on your right).

Needless to say, the similarity is… obvious. Ok, maybe not so obvious when you look at raw R, G, B numbers, scratching your head and thinking what to do with them, and there are time-tested HSV solutions waiting for you all over the internet… So, inspired by this little discovery, I went straight to code. My 1st thought was to simply project target RGB color onto this “ortho-white” plane (btw, does it have some kind of real name, anyone?), thus obtaining three-dimensional ”hue” vector in RGB, and then use ColorMatrixFilter to compute dot product with that projected vector. It resulted into this kind of picture:

Almost there, but no cigar. Thresholding this image has an effect of cutting a slice along hue wheel chord. In other words, it tends to select irrelevant colors. So I had to try again, this time with two different dot products to cut a sector-shaped slice and somewhat limit selected colors range:

This worked somewhat better and, thanks to automatic color clamping, I got yellow blobs ready to be tracked out of single ColorMatrixFilter:

In this case, just like like with hue-based approach, you get a lot of noise from grey pixels near cube diagonal. So in the end I had to combine it with 1st approach using remaining free blue channel, and got myself white blobs to track:

Well, that’s about it for the post. Check out the code, come back with your own ideas :) To sum up, RGB space is just as good as hue-based space for color matching and, since that’s the space flash is using, there is absolutely no reason to ignore it.

After December 7th, qtrack is the only free, as in “free beer”, flash 3D AR tracker. Curious why? There are many reasons for that. For one, most of its code is already available for free elsewhere on the web. But another, more important reason is that I just don’t see enough flash AR around, and setting this thing free is my modest contribution to spark some more creativity in the field :) I mean, when was the last time you actually saw flash AR game, huh?..

Any way, back to qtrack – it works similarly to famous FLARToolKit, except it is not that advanced and currently implements only single instance of single quad marker tracking. On the bright side, it is less restrictive regarding marker shape – all these markers could be tracked with qtrack, for example:

So, this is something you can try today. Zip file comes with two simple examples for Away Lite and Alternativa engines, and soon short video tutorial will follow. In January I will try to further improve qtrack in the areas where it currently have problems, and release stable version 1.0.

Credits

As I said, most of this thing is based on free code snippets published or inspired by various people. This includes blob detection (Tomek), homography (Zeh), marching squares (Sakri), Otsu thresholding (Andrew), automatic thresholding (some unknown guy), brilliant camera activity trick (Deepanjan). Even my complex numbers experiment found its way into this – I used it for pose estimation. So big thanks goes to all these people who have posted right things at right time for me to come across – without you qtrack would never happen!