<story><title>Improving compression with a preset DEFLATE dictionary</title><url>https://blog.cloudflare.com/improving-compression-with-preset-deflate-dictionary/</url></story><parent_chain></parent_chain><comment><author>polyfemos76</author><text>We use this technique in Just Dance Now, and I wholeheartedly recommend doing it for usecases where you need to compress lots of small packets individually. We save at least 50% data.&lt;p&gt;For our game, this consists of the normal communication going back and forth between server and client, where individual messages are normally &amp;lt;100b.&lt;p&gt;By sampling some user sessions we built a dictionary, fed it into deflateSetDictionary&amp;#x2F;inflateSetDictionary, and use that in both Node modules for the server, as well as throughout the ecosystem of Android and iOS.&lt;p&gt;It&amp;#x27;s very simple to do; we base all communication on standard JSON (for ease of portability and debugging) but then have the preset-dictionary fed into zip (a simple large string!) to counter the weight of JSON.&lt;p&gt;Needless to say, binary packing the protocol would give additional gains; but the productivity gains with using dictionaries to get good compression of pure JSON was a great choice for us. And it&amp;#x27;s really &lt;i&gt;supersimple&lt;/i&gt; to do.&lt;p&gt;One nice benefit is that the protocol is extendable by any developer without worrying about the wire-transport; the compression won&amp;#x27;t break is someone decides to add a parameter to a message, it just most likely won&amp;#x27;t be compressed as well as the rest of the message.</text></comment>
<story><title>Improving compression with a preset DEFLATE dictionary</title><url>https://blog.cloudflare.com/improving-compression-with-preset-deflate-dictionary/</url></story><parent_chain></parent_chain><comment><author>JoeAltmaier</author><text>Hey, somebody has a patent on pre-loading the compression tables with frequently-encountered strings. So heads up.&lt;p&gt;It was patented in the context of headers for text messages, which are sent for every message, are generally larger than the message but appear exactly once in each text so never end up in the compression table.  Each side of the conversation runs the zip algorithm on the header strings, discards the output up to that point, then compress the text message from that point. On receive they preload the algorithm with the checkpointed data structures from the preload-compression.&lt;p&gt;This Google technique is probably different enough to be off their radar. But who knows.&lt;p&gt;I think its the guys who do DirectPC</text></comment>