<story><title>Zsync: Differential file downloading over HTTP using the rsync algorithm (2010)</title><url>http://zsync.moria.org.uk/</url></story><parent_chain></parent_chain><comment><author>blaz0</author><text>I like zsync quite a bit. The chunking patcher we wrote for League of Legends is similar to zsync in some key ways.&lt;p&gt;One of the things we did to improve the efficiency of the core rolling hash based chunking logic was to add file type aware chunking. Basically, we teach the patcher about the different archive formats that our games use (say, Unreal Engine PAKs) and the algorithm ensures that chunks stop at the file boundaries within the archive, so a chunk doesn’t straddle two unrelated entries. That way when files move around in the archive we can more reliably match their chunks across versions. I think this helped improve download sizes by 5-10% in our case.</text></comment>
<story><title>Zsync: Differential file downloading over HTTP using the rsync algorithm (2010)</title><url>http://zsync.moria.org.uk/</url></story><parent_chain></parent_chain><comment><author>dividuum</author><text>zsync is great. I&amp;#x27;m currently using it to deliver system updates to Raspberry Pis. The system update (and initial install file) is a single install.zip file that gets extracted into the SD boot partition. zsync helps greatly with regards to transfer sizes as a normal OS upgrade is usually between 3-5MB instead of the full 50MB download for the full ZIP.&lt;p&gt;Of course for that you have to specially prepare how to store the files within the ZIP: For example the kernel image is stored, not compressed at it itself is already compressed. gzip based files (like initramfs.gz) are created with the rsyncable flag. And so one for other file types.&lt;p&gt;A really nice feature of zsync is that you can add any number of local data sources when syncing. It then uses all those to search for matching local data before retrieving it remotely. All without any explicit local state or special server side preparations. I usually add the previous and current OS install zip, the currently extracted OS files and previous interrupted transfers. zsync then magically uses all those to reduce download size as much as possible. This also means that the OS can initially reconstruct its own full install.zip file by using its extracted files and fetching the few remaining data chunks remotely.</text></comment>