tml'); var encodings = new Set(accepts(req).encodings()); if (encodings.has('br')) { res.setHeader('Content-Encoding', 'br'); fs.createReadStream(filename).pipe(brotli()).pipe(res); } else if (encodings.has('lzma')) { res.setHeader('Content-Encoding', 'lzma'); fs.createReadStream(filename).pipe(lzma()).pipe(res); } else if (encodings.has('gzip')) { res.setHeader('Content-Encoding', 'gzip'); fs.createReadStream(filename).pipe(gzip()).pipe(res); } else { fs.createReadStream(filename).pipe(res); } }; var certs = { key: fs.readFileSync('./https-key.pem'), cert: fs.readFileSync('./https-cert.pem'), }; https.createServer(certs, onRequest).listen(3000);

Then we can navigate to https://localhost:3000 in our browser. Let’s see what happens when I visit the server in various browsers.

Firefox 45 uses Brotli:

Opera Beta 33 uses lzma:

Safari 9 and Firefox 41 use gzip:

We can compare the size of the asset before and after compression using Firefox Developer Tools, under the network tab, by comparing the Transferred vs Size columns. The transferred column shows the bytes of the compressed content transferred over the wire, and the size column shows the asset’s decompressed size. For content sent without any form of compression, these two should be the same.

We can also verify using the curl command line utility:

$ curl https://localhost:3000 --insecure -H 'Accept-Encoding: br' -w '%{size_download}' -so /dev/null 1333 $ curl https://localhost:3000 --insecure -H 'Accept-Encoding: lzma' -w '%{size_download}' -so /dev/null 1502 $ curl https://localhost:3000 --insecure -H 'Accept-Encoding: gzip' -w '%{size_download}' -so /dev/null 1408 $ curl https://localhost:3000 --insecure -w '%{size_download}' -so /dev/null 3484

Notes about compression vs performance

The choice of which compression scheme to use does have implications. Node.js ships with zlib, but including native node add-ons for lzma and brotli will slightly increase distribution size. The time it takes the various compression engines to run can vary wildly, and the memory usage while compressing content can hit physical limits when servering numerous requests.

In the previous example, you might have noticed that lzma did not beat gzip in compression out of the box, and brotli did only maginally. You should note that all compression engines have numerous configuration options that can be tweaked to trade off things like performance for memory usage, amongst other things. Measuring the change in response time, memory usage, and Weissman score is something we’ll take a look at next.

The following numbers were gathered from running

$ /usr/bin/time -l node server.js & $ wrk -c 100 -t 6 -d 30s -H 'Accept-Encoding: <either br lzma gzip or none>' https://localhost:3000 $ fg ctrl-c

The following measurements were taken on the following machine: Early 2013 Apple MacBook Pro OSX 10.10.5 16GB 1600 MHz DDR3 2.7 GHz Core i7 4-Core with HyperThreading.

Compression Method Requests/Second Bytes Transferred (MB/s) Max RSS (MB) Avg. Latency (ms) br-stream 203 0.25 3485.54 462.57 lzma 233 0.37 330.29 407.71 gzip 2276 3.44 204.29 41.86 none 4061 14.06 125.1 23.45 br-static 4087 5.85 105.58 23.3

Some things to note looking at the numbers:

There’s a performance cliff for requests/second for compression other than gzip.

There’s significantly more memory usage for compression streams. The 9.8 GB 3.4 GB peak RSS for brotli looks like a memory leak that’s been reported upstream (my monocle popped out when I saw that).

3.4 GB peak RSS for brotli looks like a memory leak that’s been reported upstream (my monocle popped out when I saw that). The latency measured is only from localhost which would be at least this high across the Internet, probably much more. This is the waiting timing under Dev Tools > Network > Timings.

If we compress static assets ahead of time using brotli built from source, we get fantastic results. Note: we can only do this trick for static responses.

Serving statically-brotli-compressed responses performs as well as serving static uncompressed assets, while using slightly less memory. This makes sense, since there are fewer bytes to transfer! The lower number of bytes transferred per second makes that variable seem independent of the number of bytes in the file to transfer.

For compressing static assets ahead of time, we can build brotli from source