D3 Round Two: How to Blend HTML5 Canvas with SVG to Speed Up Rendering

Soon after the publication of "Digging Into D3 Internals to Eliminate Jank," I was pleased to see that it had sparked a discussion on Twitter, with D3 community members, notably Noah Veltman and Mike Bostock, sharing suggestions for improving our rendering solution:

A suggestion we received both in this discussion and on lobste.rs was to use canvas to render the data points. We had originally avoided canvas because of time constraints, lack of team familiarity with canvas, and the complications it introduced with regards to mouse interactions. However, Noah proposed a combination of SVG and canvas that strikes a balance between canvas' performance and SVG's convenience, complete with a demo:

It piqued my interest, and so I decided to explore it in some more detail here.

Combining D3 and Canvas

Our first goal is to get the same basic chart we had in the last blog post: 10,000 randomly generated points of data. We're going for something that looks like this:

When we used SVG to implement our solution, D3's data-binding was a huge draw, because it automatically associates each data point with a DOM element in our SVG. In this post, we'll use canvas to generate the points so our drawing won't be composed of DOM elements; that means we'll have to get creative to take advantage of D3's features. Irene Ros at Bocoup wrote a great post in which she detailed three methods for using canvas with D3, each method accommodating more of D3's functionality than the previous one. Since we don't need D3's data binding, we can use the first and simplest one: skip D3 altogether when drawing the points.

Let's step through a function that, given a canvas and some data, sets it up for drawing:

function paintCanvas(canvas, data) {
    // get the canvas drawing context
    const context = canvas.getContext('2d');

    // clear the canvas from previous drawing
    context.clearRect(0, 0, canvas.width, canvas.height);

    //...

The function then paints a circle for each point. Note that we assume our data comes with x and y values between 0 and 1, so we need to scale these values by the canvas' height and width.

//...
    // draw a circle for each datum
    data.forEach(d => { // start a new path for drawing
        context.beginPath();

        // paint an arc based on the datum
        const x = d.x * canvas.width;
        const y = d.y * canvas.height;
        context.arc(x, y, 2, 0, 2 * Math.PI);

        // fill the point
        context.fill();
    });
}

The outcome is pretty snappy... much faster than an implementation using pure SVG. On my laptop it comes in at approximately 22ms -- about 5 times faster than the equivalent SVG rendering. Now that we've got points on the page, we'll add in axes and mouse events, giving us rough feature parity with Cloud Manager's actual implementation.

Adding in the axes

We've forgone D3 for drawing our points, but that doesn't mean we have to live without its nifty built-ins, particularly its scales and axes. Using those will allow us to replace the manual scaling of the x and y values in our canvas painting function with calls to D3:

// Create scales
const scaleX = d3.scale.linear()
    .domain([0, 1])
    .range([0, width]);

// Create axes
const xAxis = d3.svg.axis()
    .scale(scaleX)
    .orient('bottom');

// ... and do the same to create scaleY and yAxis

The axes will be created inside an SVG, which can be absolutely positioned with CSS to overlap the canvas, like so:

Now we can add parameters for the scales into our paintCanvas function and use them to determine the x and y coordinates of our circles.

function paintCanvas(canvas, data, scaleX, scaleY) {
    // ...
    data.forEach((d) => {
        // ...
        // paint an arc based on the datum and scales
        context.arc(scaleX(d.x), scaleY(d.y), 2, 0, 2 * Math.PI);
        // ...
    });
}

With this method, we're able to take advantage of D3's axes without having to translate them to canvas. Using SVG for the pieces that aren't element-intensive lets us keep our code simple and use as much of D3's built-in functionality as possible.

Handling mouse events in Canvas

In the Cloud Manager Visual Profiler, when a user hovers over a point, the graph highlights that point and displays additional information.

Since there are no individual DOM elements in our canvas-based graph, we can't take the traditional approach of using mouseover and mouseout events. Instead, we have to listen for mouse events on the canvas element and then use the location of each event to determine the point on the canvas with which it is associated.

We start with a mousemove handler to detect when the mouse is in motion on top of the canvas element. We then use D3's mouse function to get the mouse's position on the canvas. Finally, we need to find the datum and point associated with that position (if one exists).

canvas.on('mousemove', function() {
    const mouse = d3.mouse(this);
    const x = mouse[0][0]; const y = mouse[1][1];

    // get the datum to highlight
});

It is difficult to find the right data point efficiently. Our data consist of (x, y) coordinate pairs, but the circles drawn on the graph occupy more than just one pixel. That means we can't just look at each datum to find the one with the same coordinates as the mouse, because the mouse's location may not be an exact match for the data point. What we need is a map of some sort that, given a location, can tell us exactly which datum is there.

We'll use a technique that I'll call a virtual canvas: an identical in-memory copy of our on-screen canvas. We won't render this copy to the screen, but it will contain all the same circles as our rendered canvas. The key difference is that each circle we draw on the virtual canvas will have a unique color, corresponding to exactly one of the graph's data points. When a mouse event occurs, we'll look at the virtual canvas' color in its corresponding location to determine which datum is plotted there, like this:

Noah Veltman has his own example of this approach and we'll build something very similar. We already created our on-screen canvas, so let's move on to the virtual canvas.

Creating a virtual canvas

We'll start by creating a new canvas element without appending it to the DOM.

const virtualCanvas = d3.select(document.createElement('canvas'));

We'll pass this virtual canvas into our draw function and set it up for drawing just like the original canvas.

function paintCanvas(canvas, virtualCanvas, data, x, y) {
    /*_original canvas painting here_*/

    // paint to virtual canvas
    const virtualContext = virtualCanvas.getContext("2d");
    virtualContext.clearRect(0, 0, virtualCanvas.width, virtualCanvas.height);

    /* draw points here */
}

When we draw the points on the virtual canvas, we need to assign them unique colors. (Remember, these colors will be the keys to our map later on so that we can track down the data!) For now, we'll assume we have a getColor function that handles the complexity of getting a unique color given a unique number. We can use the index of each datum in the list as our unique number.

// ...
    data.forEach((d, i) => {
        const color = getColor(i);
        virtualContext.fillStyle = color;
        virtualContext.beginPath();
        virtualContext.arc(x(d.x), y(d.y), 2, 0, 2 * Math.PI);
        virtualContext.fill();
    });
// ...

Now that our virtual canvas is set up, we can move on to our color-to-data map.

Building a color-to-data map

Now we need to fill in our getColor function. Colors in an RGB system are represented by three numbers, where each number is one byte large (0-255). If we think of these three bytes as forming a single number, every color can be represented by a single number between 0 and 256 3 . To find the color given a number in this range, we reverse this process: think of the number in binary and extract the three bytes into the Red, Green, and Blue values of a new color. To make this easier, we use d3.rgb which converts the color to a string for us.

/*
 * We're doing some bit-shifting here to more clearly illustrate how
 * to derive a color from a number, but you could accomplish the same
 * thing using modulo arithmetic and division! Check out the examples
 * to see an alternative approach
 */
function getColor(index) {
    return d3.rgb(
            (index & 0b111111110000000000000000) >> 16,
            (index & 0b000000001111111100000000) >> 8,
            (index & 0b000000000000000011111111))
         .toString();
}

Now we have to store the appropriate data in a map. We'll call our map colorToData and define it outside of our function's scope so that it will be available outside of this call.

let colorToData = {};
function paintCanvas(canvas, virtualCanvas, data, x, y) {
    // ...
    data.forEach((d, i) => {
        const color = getColor(i);
        colorToData[color] = d;
        /*_draw the points here_*/
    });
}

Now that we have a way to identify our points, we have to fill in our mouse handler. We can start by getting the RGB value of the virtual canvas at the location of the event using the rendering context's getImageData function. The data attribute of the returned object contains the RGBA color at that location, which we can feed into d3.rgb to get the string representation.

const imageData = virtualCanvas
    .node()
    .getContext('2d')
    .getImageData(mouseX, mouseY, 1, 1);

const color = d3.rgb.apply(null, imageData.data).toString();

We can read the datum we want out of our colorToData map now, but there's one last oddity we'll have to account for. Because of the effects of anti-aliasing, the edges of the circles might not be the exact color we have in our map. To combat this, we'll make sure that the datum we've found is within two pixels of the event that occurred.

const possibleDatum = colorToData[color];

if (!possibleDatum
    || Math.abs(x(possibleDatum.x) - mouseX) > 2
    || Math.abs(y(possibleDatum.y) - mouseY) > 2) {
    return;
}

Lastly, we have to repaint the canvas to show the selected datum as highlighted. Since this builds on concepts discussed earlier, we won't go into detail here, but you can view the full code sample to see the results.

And...back to SVG

There's one last problem with this approach. On my machine, it takes a full 80ms to render updates to the canvas - there's a lot of color picking and drawing to do! But we definitely don't want jank while the user is interacting with our scatterplot. So instead of repainting the whole canvas on every event, we'll overlay an SVG circle on the canvas to highlight the point.

Setting the stage

In order for this to work, the SVG must be appended to the DOM after the canvas so that it lays on top. This is easy to do: since we're using D3 to create these elements in our example, we'll just reorder the calls to append so that the SVG is added to the DOM after the canvas.

The other important step is to prevent the SVG, which is now on top of the canvas, from intercepting the mouse events that we use to determine which point is being hovered. The pointer-events CSS property is exactly what we need. It will cause all mouse events to pass through the SVG to the element underneath it when set to none.

svg {
    pointer-events: none;
}

Moving the SVG circle

Now that we've taken care of the mundane details, it's time to create the circle we'll move around to highlight the selected point. We'll create an SVG circle called highlight, select it, and keep it hidden for now using the visibility attribute.

const highlight = highlightGroup.selectAll('circle')
    .attr('r', 4)
    .attr('stroke-width', 4)
    .attr('stroke', d3.rgb(0, 190, 25))
    .attr('visibility', 'hidden');

Instead of repainting the entire canvas in our mouse handler, we'll position and show our highlight element. In the event that there is no match for the current mouse location, we hide the element.

if (mouseOutsideRange(possibleDatum, mouseX, mouseY)) {
    highlight.attr('visibility', 'hidden');
    return;
}

highlight
    .attr('cx', x(possibleDatum.x))
    .attr('cy', y(possibleDatum.y))
    .attr('visibility', 'visible');

To ensure complete functionality, we also listen to the mouseout event to hide the highlight circle when we're no longer interacting with the chart.

canvas.on('mouseout', () => {
    highlight.attr('visibility', 'hidden');
});

Now we're all set! The final result is snappy: we get all the benefits of canvas along with quick updates via SVG.

An iterative process

Our solution is far from complete; were this a production implementation, we would need to include some tweaks and optimizations. To start, we could break our canvas painting across multiple setTimeout calls like we did in our last post to eliminate jank from the initial canvas render.

Furthermore, our approach isn't the only approach out there. As we have experienced firsthand, the community has a wealth of information and inspiration for new and creative solutions. For example, Mike Bostock himself pointed out that you could skip the color to data map and use a quadtree or voronoi diagram to map points on your canvas back to your data.

Although it may be a little while until a refactor this big hits our Visual Profiler, we're always learning from the community, and we'll keep sharing our approaches to improving data visualization in the browser.