<story><title>Honey, I shrunk the NPM package</title><url>https://jamiemagee.co.uk/blog/honey-i-shrunk-the-npm-package/</url></story><parent_chain></parent_chain><comment><author>terrelln</author><text>In parts (1) and (2) comparing the default setting of Zstd (level 3) against the default setting of Brotli (level 11) is a bit misleading. It shows Brotli compressing ~30% better than Zstd, but Brotli&amp;#x27;s default level is &amp;gt;100x slower than Zstd&amp;#x27;s default level. Zstd level 3 is expected to run at hundreds of MB&amp;#x2F;s, and Brotli level 11 is expected to run at ~2 MB&amp;#x2F;s. The compression speed is only 30% slower because that benchmark includes the time to tar the directory, which is likely more expensive than the compression itself. As @sfink already suggested, just running lzbench on the npm-9.7.1.tar would be a better benchmark.&lt;p&gt;In part (3), because its running only on lib&amp;#x2F;npm.js which is 13KB, you are getting skewed results which aren&amp;#x27;t directly applicable to the compression of npm-9.7.1.tar. Brotli excels at compressing small Javascript files, as this is where its dictionary provides the most benefit. The benefits of the dictionary for a large tar file will be negligible.&lt;p&gt;However, in the npm-9.7.1.tar scenario we still expect Brotli level 11 to produce slightly smaller files than Zstd level 19. Likely ~5% smaller. But we do expect Zstd to provide significantly faster decompression speed.</text></comment>
<story><title>Honey, I shrunk the NPM package</title><url>https://jamiemagee.co.uk/blog/honey-i-shrunk-the-npm-package/</url></story><parent_chain></parent_chain><comment><author>sfink</author><text>&amp;gt; The first [caveat] is that lzbench isn’t able to compress an entire directory like tar , so I opted to use lib&amp;#x2F;npm.js for this test.&lt;p&gt;As opposed to... just using the npm-9.7.1.tar file that the other tests were just using (sorta barely internally to tar, but I don&amp;#x27;t think tar does any fancy streaming or anything if you&amp;#x27;re passing something via --use-compress-program, certainly nothing that would skew the results more than replacing all of npm.tar with just npm.js.)&lt;p&gt;In my local install, npm.tar is 25MB. npm.js is 16KB. It may not change the final outcome, but the data in the article do not support the conclusion. I would strongly suggest tarring up the npm directory and rerunning lzbench.</text></comment>