How to set object.matrix in three.js?

While all the top google hits for this problem revolve around messing with matrixAutoUpdate property, I thought I’d share this simple trick:

object.matrix.copy (...); // or fromArray (...), etc
object.matrix.decompose (
    object.position,
    object.quaternion,
    object.scale
);

This only really works with the matrices that can be represented with three.js’ position/quaternion/scale model, but in practice this is vast majority of the cases. You’re welcome.

Using VideoTexture with Camera in AIR

These days I am trying to write simple iOS app that takes live camera stream and applies custom shader to it, as my pet project. And – surprise – I am doing this with AIR. That’s right, after I have finally come to terms with the fact that flash is dead and, considering how much GLSL is superior to AGAL as a shader language, I am still using AIR to build this app. Why? Because it works :) Unlike HTML5 aka flash killer. WebRTC is still not supported on iOS, and community polyfills simply are not enough to plug the gap. So, the only other option would be to use ObjC or Swift, but I know neither yet, while with flash I am at home (and, potentially, can easily port the app to Android later).

VideoTexture ?

Ok, so we have simple access to live camera feed, but how to bring it to Stage3D (flash version of WebGL) to apply the shader? Traditionally, we had to draw the display list video object into bitmap and re-upload it to GPU every frame, which was costly. But quite recently Adobe gave us more straightforward option – VideoTexture. Whole new runtime feature with amusing history, however, flash is still dead, so its documentation is crap. Just following NetStream-based snippets and changing attachNetStream to attachCamera like this

var videoTexture:VideoTexture = stage3D.context3D.createVideoTexture ();
videoTexture.attachCamera (camera);
videoTexture.addEventListener (Event.TEXTURE_READY, renderFrame);

did not work.

The missing piece ?

There is RENDER_STATE event that can tell you if video decoding is GPU accelerated (in the case of live camera stream it is not, btw). Turns out, you have to subscribe to this event even if you don’t care, or else the magic does not happen – camera stream data will never reach the texture and, correspondingly, TEXTURE_READY events will never fire. You’d think this is stupid and can’t possibly be true, but then adding event listeners is exactly how you get webcam activityLevel property to work, for example, so I am not surprised. With this, the final code to make it work is

var videoTexture:VideoTexture = stage3D.context3D.createVideoTexture ();
videoTexture.attachCamera (camera);
videoTexture.addEventListener (VideoTextureEvent.RENDER_STATE, function (e:*):void {
	// yes, we want TEXTURE_READY events
});
videoTexture.addEventListener (Event.TEXTURE_READY, renderFrame);

Center element with unknown width

As with bootstrap progress bar post, this one is actually about CSS, not javascript, but I don’t want too many tags. So, the subject of today’s tutorial is centering stuff. In the good old days we could just place its left side at 50% and set left margin to minus half of content width. But what if we do not know content width in advance and therefore can’t set the margin a priori? That’s where newer CSS stuff helps:

transform: translate(-50%, -50%);

Yes, those are percents of its actual dimensions, whatever they will be. You can see it in action here.

Three.js and 2D overlays

How do you handle the problem of 2D overlays in three.js-based app? You can go for camera-oriented planes, I guess, or maybe use sprites, but with just a few of them overlays why not just use DOM? The trick is to wrap the canvas in overflow:hidden element with display:relative or absolute, and place your overlays in there. This method has several problems that needs to be addressed.

When do you show and hide the overlays?

Object3D add() and remove() methods dispatch ‘added’ and ‘removed’ events, but this is only part of the story. You have to check if other objects obscure your overlay anchor, or else it will incorrectly show up on top of those objects. In simple cases, you could calculate this analytically, but general solution would have to rely on something like ray casting.

Can these things be calculated implicitly?

Yes they can, by overriding Object3D’s updateMatrixWorld() method, which is called on every object when you render the scene. Of course, you still have to call original method, too.

The above method implementation will look like this. As always, feel free to discuss this stuff in comments.

Three.js sprites and custom shaders?

Do three.js sprites support custom shaders?

No. They have SpriteMaterial thing that holds your beer parameters for this shader. You can see that this shader is compiled once and is totally inaccessible from outside of SpritePlugin.

What do I do about this?