Posts Tagged 'augmented reality'

Still messing with qtrack

Ok, boring things first. I uploaded another qtrack version with some minor fixes and improvements, and also added broomstick example there. Nothing groundbreaking so far.

Now, to fun things. I tried to make non-square marker qtrack mod based on this method. You can see in this video how far I could push it:

but, overall, this was a failure. Unlike that guy, I had only one shape to detect, so at 1st I thought I could use max ICF value between blob and marker contours (ideally 1) and compare it to confidence threshold. However, this picked way too much false positives. So in 2nd iteration, that you see in the video, I had to calculate max ICF values for few hundred blobs 1st, and then pick the blob with max of those values. This worked somewhat better, but in the long run, it is still unreliable. Here is a distribution of max max ICF values thresholded at 0.7:

As you see, even shape as simple as flash logo produces really poor values most of the time. Another equally significant problem is that BitmapData’s threshold() method produces poor contours when threshold is not optimal: over-illuminated patches of the marker punch random “holes” in its binarized version and contaminate detected contour. To sum up, you will have to stick with quad markers for some time, sorry.


Update on qtrack

For those of you who are still interested in my free marker-based flash AR tracker, I updated qtrack to v1.0. There were actually only few minor changes, however. 1st, I changed detector code to avoid v0.9 marching squares bug; interesting side effect of this change is that now literally anything that remotely resebles a quad can be used for the marker (like those examples on the right). 2nd, I added requested option to track the marker in NetStream-ed video. 3rd, there is new makePlaneTransform method that should help you with molehill setup (but no molehill demo included for now). 4th, some minor stuff you wouldn’t care about.

On a related note, back in December I promised to make Alternativa3D AR video tutorial for russian xmas, that would include qtrack and other trackers available through FLARManager. I did not finish that video in time, so for this post I uploaded qtrack part only (and chances are that FLARManager part will be only available in text version of tutorial). The video is in russian, but you can try english subtitles. Enjoy:

Meet qtrack

It stands for “quadrilateral tracker”, little pet project I have been sporadically working on since october. I should have given it more love, but I didn’t want this post to drag into next year, so here goes “version 0.9”:

After December 7th, qtrack is the only free, as in “free beer”, flash 3D AR tracker. Curious why? There are many reasons for that. For one, most of its code is already available for free elsewhere on the web. But another, more important reason is that I just don’t see enough flash AR around, and setting this thing free is my modest contribution to spark some more creativity in the field :) I mean, when was the last time you actually saw flash AR game, huh?..

Any way, back to qtrack – it works similarly to famous FLARToolKit, except it is not that advanced and currently implements only single instance of single quad marker tracking. On the bright side, it is less restrictive regarding marker shape – all these markers could be tracked with qtrack, for example:

So, this is something you can try today. Zip file comes with two simple examples for Away Lite and Alternativa engines, and soon short video tutorial will follow. In January I will try to further improve qtrack in the areas where it currently have problems, and release stable version 1.0.


As I said, most of this thing is based on free code snippets published or inspired by various people. This includes blob detection (Tomek), homography (Zeh), marching squares (Sakri), Otsu thresholding (Andrew), automatic thresholding (some unknown guy), brilliant camera activity trick (Deepanjan). Even my complex numbers experiment found its way into this – I used it for pose estimation. So big thanks goes to all these people who have posted right things at right time for me to come across – without you qtrack would never happen!

Augmented reality and QR codes

This is basically proof-of-concept that this is already possible in flash, today. What you do is take saqoosha’s FLARToolKit, add keno’s QR code reader, and load results into 3d engine (in this case, kiroukou’s sandy).

Clicky to see PoC

It doesn’t require webcam, just click to upload your test image. It also makes use of saqoosha’s PixelBender homography transform filter, since keno’s reader apparently doesn’t do that on its own (and generally is very sensitive to input bitmap, I must add). Default image comes from here.

Alternative to adaptive thresholding

As cool as it is, adaptive thresholding generates too much redundant data. The only alternative implemented before seems to be the method called automatic thresholding, suggested by ARToolKitPlus folks:

ARToolKitPlus can do dynamic thresholding by looking at the marker content (pattern) and taking the average between the darkest and brightest pixels. If no marker is found the threshold value is randomized.

From this description alone, it is clear that there will be frames with marker undetected even though it is clearly visible in the image, so I thought, until now, that this method has to be quite bad. It turns out, however, that this method actually works very well and the number of lost frames is very small. In practice, you don’t even have to find pixel values range, it suffices to assume that current good threshold value lies close to last known one.

On code side, you no longer need to write or import some fancy filter classes, all you have to do is to modify your fixed thresholding main loop just a bit:

private var threshold:Number = 0.5;
private var confidence:Number = 0, confidenceThreshold:Number = 0.5;
private function onEnterFrame (e:Event):void {
	videoSnapshot.draw (video);

	if (confidence < confidenceThreshold) {
		// randomize threshold
		threshold *= 0.8;
		threshold += 0.2 * Math.random ();

	confidence = 0;
	if (detector.detectMarkerLite (raster, 255 * threshold)) {
		confidence = detector.getConfidence ();

		detector.getTransformMatrix (result);
		stuff.setTransformMatrix (result);

	scene.render ();

If you’re sandy fan, you can play with this method using this stub code.

Augmented reality based on blob detection

It’s long time no post now, so I thought why not post something :) So, here goes this little augmented reality experiment. Let me warn you first, it is not impressive at all, nothing fancy on 3D side, and tracking – to be fair – sucks. This is in part because it does not use any solid technology such as FLARToolKit, just something hastily hacked in haXe, for a change. In other part, it uses very simple fixed threshold filter, that can (and does) harm tracking. To save me some typing, I have made this short video explaining what this experiment is about:

Continue reading ‘Augmented reality based on blob detection’

FLARToolKit adaptive filter experiment sources

For those who asked about sources of my little experiment from last week, here they are. Make sure you are using FLARToolKit revision 2570 with these.

If you wasnt reading FLARToolKit mailing list, here is short summary: FLARToolKit converts color image to “binary” image before doing any magic on it, using fixed threshold filter. In fact, this threshold is simply hard-coded to be 80 in many examples, and you will need to write more code to detect good values for threshold if you want your application to run regardless of lighting settings. The filter in this experiment avoids the problem by using local average brightness as a threshold for every pixel; the downside is that it enables FLARToolKit to “see” far more details, thus wasting even more CPU time. As a work-around, I tried to narrow down search area in the image using last detected square, but this only helps when marker motion is limited.

Well, check it out, play with it, and let me know what you think.

Old stuff

September 2018
« Jan    

Oh, btw…