Transparent video texture in three.js

Some weeks ago I was asked if it is possible to have transparent video texture in THREE.js and, although I was sure it’s totally possible, I could not find any examples of it. Today I am sharing one such example with you, so you don’t have to code it yourself in case someone will ask you the same question :)

Ladies

This demo is using good old “green screen” trick to create transparency in otherwise opaque texture. You can find similar examples out there, but they are neither THREE.js- nor GPU- based.

So, where do you start? I started here – you can easily tell how much code was borrowed from Lee Stemkoski. Basically, I only had to replace MeshBasicMaterial with ShaderMaterial. Creating custom material sounded scary at first, but turned out to be really easy in the end thanks to helpful tutorials (such as this one). Finally, the shader – I want it to be as simple as possible, so no fancy color conversion stuff in there – it operates directly in RGB using 1st method from my earlier post (err, no, it’s different method). The result is basically two lines of code:

vec3 tColor = texture2D(texture, vUv).rgb;
gl_FragColor = vec4(tColor, (0.9 - dot(color, tColor) / (length(tColor) + 0.05)) * 3.0);

Let’s break 2nd line down. It starts at the dot product of colors – the dot product approaches 1 when texture color approaches the color we want to remove. In theory, it should have been dot (normalize(color), normalize(tColor)), but normalize(tColor) part fails on pure black pixels (such as girl hair), so I had to use tColor / (length(tColor) + small number) instead. This handles black colors better, but now the dot is always less than 1 – hence you see 0.9 in there. You can also see that I do not normalize “color” parameter – there is no good reason to do it inside the shader for every pixel. Finally, 3.0 is randomly chosen magic number that determines how fast texture colors become transparent as they approach the color to remove – check out this graph to see its effect.

Update: I have just made the shader even simpler using length(tColor – color). Initially I thought absolute color values will not work well, but the actual test says they do:

Angle-based shader on the left, distance-based shader on the right.

Angle-based shader on the left, distance-based shader on the right.

3 Responses to “Transparent video texture in three.js”


  1. 1 samantafluture April 10, 2014 at 20:57

    Awsome!

    I was wondering if it’s possible to use the webcam capture as ‘video.src’, instead of an ‘url’, and use it as a texture, much like in the same way you do.

    I’ve tried to use the threex.videotexture lib, but I had no success.

    Sorry if it’s a dumb question, but I’m quite new to Three.js…

    I am building an instalation and it requires to subtract the background of a live video. And this live video will be using a colored background, similar to greenscreen/chroma key.

    Thanks a lot


  1. 1 Шанс для javascript программиста познакомится с WebGL | Sergey Kurdakov blog Trackback on July 12, 2014 at 07:02

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




Old stuff

April 2014
M T W T F S S
« Mar   May »
 123456
78910111213
14151617181920
21222324252627
282930  

Oh, btw…


Follow

Get every new post delivered to your Inbox.

%d bloggers like this: