3D bar chart on the globe using three.js

When people need to present per-country data such as GDP, they usually resort to color-coded maps or simple bar charts. In 3D, you could combine these two and end up with something like this:

globe bar chartThis is very obvious idea, yet I had no time to try it until last November. It worked, and I went on with my life, leaving the code to collect the dust. Then I saw Kaspersky cyberthreat map this March, which used extruded country shapes to support zooming. This reminded me this gold I was sitting on, and now I was finally able to fork out some time to undust it and push to github.

Overview

Basically, you take 2D country shapes and tessellate it on the sphere surface, then extrude towards the sphere center. You end up with cone-shaped meshes that you can then scale to build this kind of geo- bar chart. The catch is that corresponding three.js scene weighs much more than its 2D source data (ObjectExporter blows 100 KB of GeoJSON up to many MBs). Much of this extra weight comes in the form of useless digits in vertex data, but even with that removed there is extra dimension, data duplication and so on. To work around this issue you have to build 3D geometry from tessellated 2D data on the fly – this puts 3D data size at ~150% of 2D. Still sucks, if compared to TopoJSON, but tolerable. Corresponding code looks like

for (var name in data) {
    geometry = new Map3DGeometry (data[name], innerRadius);
    globe.add (data[name].mesh = new THREE.Mesh (geometry, material));
}

Map3DGeometry class and the tool to create the data are now at github.

Tessellation

Following documents the problems you would encounter if you did it yourself, so you may probably skip it. Continue reading ’3D bar chart on the globe using three.js’

Transparent video texture in THREE.js

Some weeks ago I was asked if it is possible to have transparent video texture in THREE.js and, although I was sure it’s totally possible, I could not find any examples of it. Today I am sharing one such example with you, so you don’t have to code it yourself in case someone will ask you the same question :)

Ladies

This demo is using good old “green screen” trick to create transparency in otherwise opaque texture. You can find similar examples out there, but they are neither THREE.js- nor GPU- based.

So, where do you start? I started here – you can easily tell how much code was borrowed from Lee Stemkoski. Basically, I only had to replace MeshBasicMaterial with ShaderMaterial. Creating custom material sounded scary at first, but turned out to be really easy in the end thanks to helpful tutorials (such as this one). Finally, the shader – I want it to be as simple as possible, so no fancy color conversion stuff in there – it operates directly in RGB using 1st method from my earlier post (err, no, it’s different method). The result is basically two lines of code:

vec3 tColor = texture2D(texture, vUv).rgb;
gl_FragColor = vec4(tColor, (0.9 - dot(color, tColor) / (length(tColor) + 0.05)) * 3.0);

Let’s break 2nd line down. It starts at the dot product of colors – the dot product approaches 1 when texture color approaches the color we want to remove. In theory, it should have been dot (normalize(color), normalize(tColor)), but normalize(tColor) part fails on pure black pixels (such as girl hair), so I had to use tColor / (length(tColor) + small number) instead. This handles black colors better, but now the dot is always less than 1 – hence you see 0.9 in there. You can also see that I do not normalize “color” parameter – there is no good reason to do it inside the shader for every pixel. Finally, 3.0 is randomly chosen magic number that determines how fast texture colors become transparent as they approach the color to remove – check out this graph to see its effect.

Update: I have just made the shader even simpler using length(tColor – color). Initially I thought absolute color values will not work well, but the actual test says they do:

Angle-based shader on the left, distance-based shader on the right.

Angle-based shader on the left, distance-based shader on the right.

How to check if URL is not MP3 ?

Since flash player can’t send HEAD requests, you have to resort to dirty tricks if you want to check URL content type. In this example, we are going to try and figure out if some URL points to playable sound. Since we’re dealing with unknown URL, chances are that its domain does not permit us to access raw data, so we can’t really use URLLoader – we need to actually load whatever URL points to into Sound object. However, if you thought that you could just catch i/o error 2124, like you would do with image Loader, for example, you’re in for big surprise: it does not fire. Instead, Sound object fires progress and complete events, and is even happy to play (although you will not hear any noise, or anything at all).

So, how do we detect that our Sound object is actually broken? First thing you’ll notice is that even after complete event it is still buffering. Unusual, but the same thing happens with really short sounds, so this check is not reliable. The other thing is that extract() method obviously fails to return any samples. This in itself is again not enough, since it does happen with partially loaded sounds too. However, the combination of the two seems to hit the nail:

private function onComplete(e:Event):void {
   var s:Sound = e.target as Sound;
   if (s.isBuffering && (s.extract(new ByteArray(), 1, 0) < 1)) {
      // url does not point to playable sound
   }
}

See this method in action at wonderfl.net :)

WTF is going on in Ukraine?

This week ukrainian events boil down to this:

This is how it goes...

Continue reading ‘WTF is going on in Ukraine?’

Amazing Maps (and one in particular :)

Today I came across (and followed) Amazing Maps twitter account. They post loads of interesting maps, like the one that shows who has supplies the oil, or the one that shows that my country has world’s best extinction rate. This post, however, is dedicated to the map of countries prosecuting gays:

Personally, I don’t care if you are gay or not (unless you’re girl that I am interested in), but this map scored two points on my bullshitometer for dirty tricks it uses to influence my opinion. 1st, and obvious, was to cover offending countries in blood. Of executed gays, I suppose. 2nd, and less obvious, is clever choice of projection where gay-neutral Greenland (small island with 80% of its area being nothing but ice) has roughly the same size as whole gay-hating Africa. So, I decided to re-do this map using positive green color and population-based projection to make it a bit more representative:

Turns out it does not look that different, after all :) Gay-neutral China, the world’s most populated country, conveniently offsets all those aftrican gay haters. In fact, I’m sure that very soon indian government will sort out their bureaucratic bullshit, and half of this green spot will be gone.

Update: they have now posted new version of the map that uses better projection and breaks it up by punishment severity.

Using D3.js with THREE.js, part II

So, our little D3 + 3D experiment continues. This time we will be trying to replicate bar chart tutorial 2 in 3D.

Transitions

Two things need to be done before D3 transitions could work. First, D3 must be able to interpolate “attribute” values, so we need to implement both setAttribute and getAttribute methods in THREE.js:

// this one is to use D3's .attr() on THREE's objects
THREE.Object3D.prototype.setAttribute =
  function (name, value) {
    var chain = name.split('.');
    var object = this;
    for (var i = 0; i < chain.length - 1; i++) {
        object = object[chain[i]];
    }
    object[chain[chain.length - 1]] = value;
}

// and this one is to make'em work with D3's .transition()-s
THREE.Object3D.prototype.getAttribute =
  function (name) {
    var chain = name.split('.');
    var object = this;
    for (var i = 0; i < chain.length - 1; i++) {
        object = object[chain[i]];
    }
    return object[chain[chain.length - 1]];
}

Second, D3′s selectAll() now needs to actually select something to build updating selection. We can make it select various descendants of Object3D type, for example:

// now selectAll must actually do something
THREE.Object3D.prototype.querySelectorAll =
  function (selector) {
    var matches = [];
    var type = eval(selector);
    for (var i = 0; i < this.children.length; i++) {
        var child = this.children[i];
        if (child instanceof type) {
            matches.push(child);
        }
    }
    return matches;
}

These changes are already enough to replicate their simple transition example in THREE.js:

var t = 1297110663,
    v = 30,
    data = d3.range(9).map(next);

function next() {
  return {
    time: ++t,
    value: v = ~~Math.max(10,
      Math.min(90, v + 10 * (Math.random() - .5))
    )
  };
}

setInterval(function() {
  data.shift();
  data.push(next());
  redraw();
}, 1500);

function redraw() {
  d3.select( chart3d )
    .selectAll("THREE.Mesh")
    .data(data)
  .transition()
    .duration(1000)
    .attr("position.y", function(d, i) { return d.value; })
    .attr("scale.y", function(d, i) { return d.value / 10; })
}

But to replicate their final keyed data join example, we need to take care of one last thing…

Exit selections

We want D3′s remove() to actually remove 3D objects. Now, you might be thinking, this is easy one, we’ll just add removeChild() to Object3D prototype

THREE.Object3D.prototype.removeChild =
  function (c) {
    this.remove(c);
  }

and it should work, right? Wrong, it does not! To solve this one, I had to check D3 source. Turns out D3 does not call removeChild on selection parentNode but on node’s parentNode itself, so we need to modify our appendChild as follows:

THREE.Object3D.prototype.appendChild = function (c) {
    this.add(c);
    // create parentNode property
    c.parentNode = this;
    return c;
}

Final result

With these changes, it is now possible to do this:

function redraw() {

	var bars = d3.select( chart3d )
		.selectAll("THREE.Mesh")
		.data(data, function(d) { return d.time; });

	// move existing bars to their new place
	bars.transition()
		.duration(1000)
		.attr("position.x",
			function(d, i) { return 30 * i; })

	// add new bar and make it grow
	bars.enter().append( newBar )
		.attr("position.x",
			function(d, i) { return 30 * (i + 1); })
		.attr("position.y", 0)
		.attr("scale.y", 1e-3)
	  .transition()
		.duration(1000)
		.attr("position.x",
			function(d, i) { return 30 * i; })
		.attr("position.y",
			function(d, i) { return d.value; })
		.attr("scale.y",
			function(d, i) { return d.value / 10; })

	// remove the bar that is no longer in data set
	bars.exit().transition()
		.duration(1000)
		.attr("position.x",
			function(d, i) { return 30 * (i - 1); })
		.attr("position.y", 0)
		.attr("scale.y", 1e-3)
		.remove()
}

Final demo

This is still not complete THREE.js “plugin” but, as you can see, it’s a nice start.

Using D3.js with THREE.js

This blog name has “3D” in it, but there was no 3D here for a long time now. So I am fixing this tonight.

Once upon a time we did simple AS3 port of D3.js lib here at CodeOrchestra, codenamed “D6″ by Potapenko. The name D6 came from D3 + 3D, since our port was able to work with 3D engines to enable 3D data visualization. That port never went into production, but the idea behind it is still valid today.

I mean, can you do 3D with D3? There is no 3D examples in their gallery, so the obvious answer seems to be “no”. The reason is that D3 was built with browser DOM in mind, and it is not straightforward to extend it to support arbitrary object models such as THREE.js, de facto standard for 3D in javascript world. Nevertheless, as you can guess, it could be done.

Let’s take a look at D3 simplest example (bar chart):

d3.select(".chart")
  .selectAll()
    .data(data)
  .enter().append("div")
    .style("width", function(d) { return d * 10 + "px"; })
    .text(function(d) { return d; });

Here, a number of methods accept browser-specific magic strings such as “.chart” selector or “div” tag name. While convenient in 2D browser environment, this is totally useless for our purposes. But have no fear, for D3 comes with alternative signatures for these methods! These signatures were designed for boring things like reusing existing selections, but we can actually bend them to swallow non-DOM objects. For example, we can rewrite the bar chart above like this:

function createDiv() {
	return document.createElement("div");
}

var chart = {
	appendChild: function (child) {
		// this is called by append() after createDiv()
		return document.getElementById("chartId")
			.appendChild(child);
	},
	querySelectorAll: function () {
		// this is called by selectAll()
		return [];
	}
}

d3.select( chart )
  .selectAll()
    .data(data)
  .enter().append( createDiv )
    .style("width", function(d) { return d * 10 + "px"; })
    .text(function(d) { return d; });

Here, we 1) explicitly instruct D3 how to create “div” tags, and 2) trick D3 into thinking that “chart” is normal DOM object, while the end result of the script is exactly the same as before. Now when we learned how to work around D3 helpful APIs, we can easily do the same with THREE.js:

THREE.Object3D.prototype.appendChild = function (c) {
	this.add(c); return c;
}

THREE.Object3D.prototype.querySelectorAll = function () {
	return [];
}

chart3d = new THREE.Object3D();

d3.select( chart3d )
  .selectAll()
  .data(data)
.enter().append(
  function (d, i) {
	var bar = new THREE.Mesh( geometry, material );
	bar.position.x = 30 * i;
	bar.position.y = d;
	bar.scale.y = d / 10;
	return bar;
  }
);

In this demo, I actually went ahead and made D3′s attr() calls work for why not.

Well, this is it for today :) I hope to continue this post some day with part 2, where I’d get D3′s transitions to work, but right now I need to sleep. Zzzz..


Old stuff

April 2014
M T W T F S S
« Mar    
 123456
78910111213
14151617181920
21222324252627
282930  

Oh, btw…


Follow

Get every new post delivered to your Inbox.