Digging into D3 Internals to Eliminate Jank over Large Data Sets

Browser JavaScript isn't like most other user-facing application runtimes, where a main thread handles the UI and you can spin work off into background threads. Because it's single-threaded, when you have to do heavy lifting, sometimes you need to get creative to keep your UI responsive.

This was the case on one of Cloud Manager’s newest features, the Visual Profiler. It was an ambitious design, and when I was given the initial mocks I was immediately excited by the prospect. As a front-end engineer, I couldn’t wait to start implementing the new chart and table:

We didn’t have anything quite like it in the app, and I found out pretty quickly that our existing charts library wasn’t going to support it. Not missing an opportunity to try something new, I moved forward with a D3-based solution and got a chart going pretty quickly. Unfortunately, when I connected it to actual data from MongoDB’s built-in Profiler, it ground to a halt.

I should have realized it sooner, but the number of data points we were hoping to plot was tremendous. In D3 this is a problem because it’s built on top of the HTML5 SVG element, meaning adding and removing plot points requires a DOM manipulation. Although we do throttle the number of profiler entries we take in from a single customer’s application (no one wants to try to drink from a firehose), the Visual Profiler would still need to handle over 10,000 points. The chart was meant to be responsive, allowing users to select, filter, zoom, and change the statistics being shown, but most people’s browsers just can’t handle inserting that many DOM elements in real time. On my reasonably well-equipped 2014 Macbook Pro, using any of those features froze the browser for a good 5 seconds. Users on older computers might as well force quit their browsers.

Well, that’s what we call jank - it's what happens when you ask the browser to do more rendering than it can handle before a single frame is rendered. At 60 frames per second that’s a measly 16 milliseconds. To fix the jank, I was going to have to break up the rendering into smaller chunks that would fit inside that tiny window... and to do that, I was going to have to get up close and personal with the D3 internals.

Setting up the problem

To simplify the problem, we're going to work on code from this gist. It's about as basic as we can get to demonstrate and solve the problem. If you believe me that the problem exists, we can ignore the full gist and just look at this section of code:

function drawCircles(svg, data) {
    var circles = svg.selectAll('circle').data(data);
    circles.enter().append('circle')
        .attr('r', 3);

    circles.exit().remove();

    circles
        .attr('cx', function(d) { return d.x })
        .attr('cy', function(d) { return d.y });
}

The function takes a D3 svg selection and an array called data of (x, y) coordinates. With D3's enter() function, we can get the subset of data points that don't have matching DOM elements, and add a circle for each new point. The chart comes out like this:

This code will execute all at once when the JavaScript event loop reaches it, without yield, until all the points are rendered. If we call this code with a large enough number of data points on a button press, you'll see your browser hang before it even returns the button to its undepressed state. That's the easiest part to fix, actually, and it will lay the foundation for keeping the whole thing snappy, so let's start there.

Putting functions on the message queue

To fix the browser hang, we slot our function at the end of JavaScript's message queue. This allows the browser to completely handle the button press event and repaint the button before it executes any of our drawing code. The easiest way to do that is wrap our code in a setTimeout:

function drawCircles(svg, data) {
    setTimeout(function() {
        var circles = svg.selectAll('circle').data(data);
        circles.enter().append('circle')
            .attr('r', 3);

        circles.exit().remove();

        circles
            .attr('cx', function(d) { return d.x })
            .attr('cy', function(d) { return d.y });
    }, 0);
}

In this gist you’ll notice that the button returns to its undepressed state before the circles finish being drawn. “That was easy!” you might say, but now comes the hard part. We've simply delayed the problem a little. While we’re adding and maneuvering all those circles on the page, the browser is still going to hang, and that’s what we’re trying to avoid. So now, we’re going to have to break our D3 Selection circles into batches, and render each batch in a separate message on the queue.

Breaking a D3 selection into batches

D3 doesn’t provide native functions for breaking up the selection above (circles) into batches for rendering. In other situations, you might be able to use select() or filter() (which give you a subset of a selection) but they don’t preserve the data-binding that provides the convenient joins called update, enter(), and exit(). You might try to break up the data into batches earlier, when you give it to D3, but then D3 would compute inaccurate joins leaving you with mismatched data-binding and fewer circles than you need.

Building your own update, enter, and exit selections

A D3 selection is a subclass of an array, whose elements represent groups of DOM nodes as more arrays. Who doesn’t like an array of arrays? In our case, there is only one “group of DOM nodes” subarray, so circles[0] gets all the current DOM elements we want. We’ll also need circles.enter() and circles.exit() to have all the possible elements for our data. That’s because for each new element that is entering the data set, D3 puts a null placeholder in circles[0], and the actual element in the corresponding slot of the enter() selection. That is to say, when we bind the data to our selection, if the nth datum does not have a corresponding element in the DOM, circles[0][n] is a new slot containing null, and circles.enter()[0][n] contains an element for the new datum.

Lastly, to get the first batchSize elements as represented in the update(), enter() and exit() joins, we can slice the arrays of DOM elements and wrap them in a new selection like so:

var updateSelection = d3.selectAll(circles[0].slice(0, batchSize));
var enterSelection = d3.selectAll(circles.enter()[0].slice(0, batchSize));
var exitSelection = d3.selectAll(circles.exit()[0].slice(0, batchSize));

Out with the exit, in with the enter!

So now we’ve got our three selections, and we want to perform the same operations as before. The way we handle updates and exits doesn’t have to change because these selections refer to actual DOM elements:

// equivalent to circles.exit().remove() in the un-batched example
exitSelection.remove();

// equivalent to circles.attr in the un-batched example
updateSelection
    .attr('cx', function(d) { return d.x })
    .attr('cy', function(d) { return d.y });

But the way we handle enterSelection has to be completely different than how we handle circles.enter(). Calling enter() on a selection generates a sub-type of selection, inside of which are nulls for all existing elements and dummy objects for each new element. This special sub-type of selection knows how to handle those nulls when we call append(), but enterSelection, which was generated via selectAll(), is an ordinary selection, and will not handle nulls correctly. The good news is that the dummy objects that enterSelection contains still have the data that we bound using data(), so we can use each to append the DOM elements we need and give them the right data:

enterSelection.each(function(d, i) {
    var newElement = svg.append('circle')[0][0];
    newElement.__data__ = this.__data__;
}).attr('r', 3);

This will get us an element in the DOM for every element in enterSelection, but the other convenient advantage of D3s joins is that they automatically update one another. Specifically, when you call append() on an enter() join, the DOM elements that are created are added into the enter() join as well as the update() selection you created the enter() join from. Our enterSelection is missing this functionality because it isn’t a real enter() join, so to do that we’ll have to modify enterSelection and updateSelection manually as we go along.

enterSelection.each(function(d, i) {
    var newElement = svg.append('circle')[0][0];
    newElement.__data__ = this.__data__;
    enterSelection[0][i] = newElement;
    updateSelection[0][i] = newElement;
}).attr('r', 3);

And that’s about it. Yes, we’re reaching into D3’s internals, but we’re walking away with a highly optimized batched rendering process.

Putting it all together

The last step is to generalize this across multiple batches and split it across multiple timeouts. We do that by calculating a startIndex and stopIndex for a particular batch, and write a new drawBatch function to be called with setTimeout:

function drawCircles(svg, data, batchSize) {
    var circles = svg.selectAll('circle').data(data);

    function drawBatch(batchNumber) {
        return function() {
            var startIndex = batchNumber * batchSize;
            var stopIndex = Math.min(data.length, startIndex + batchSize);
            var updateSelection = d3.selectAll(circles[0].slice(startIndex, stopIndex));
            var enterSelection = d3.selectAll(circles.enter()[0].slice(startIndex, stopIndex));
            var exitSelection = d3.selectAll(circles.exit()[0].slice(startIndex, stopIndex));

            enterSelection.each(function(d, i) {
                var newElement = svg.append('circle')[0][0];
                enterSelection[0][i] = newElement;
                updateSelection[0][i] = newElement;
                newElement.__data__ = this.__data__;
            }).attr('r', 3);

            exitSelection.remove();

            updateSelection
                .attr('cx', function(d) { return d.x })
                .attr('cy', function(d) { return d.y });

            if (stopIndex < data.length) {
                setTimeout(drawBatch(batchNumber + 1), 0);
            }
        };
    }

    setTimeout(drawBatch(0), 0);
}

We also had to add a condition to the end of the drawBatch function that calls itself for the next batch if there is another batch to draw. If you run this gist you’ll now notice that it takes a bit longer to render, but the entire page remains responsive while it happens.

A responsive visual profiler

After all that digging through the internals of D3 and hacking together my own batched rendering, I connected the new chart to real data from our backend. Watching the scatterplot render progressively while the page remained responsive was the ultimate payoff. No jank, just a great user experience with a scatter plot that helps our users find the choke points in their MongoDB deployment.

I’m still blown away everytime I look at it! But now I'm blown away because it’s working, not because of all the rendering obstacles standing in the way of getting it done.