<story><title>Incremental Backups Using GNU Tar and S3</title><url>https://ops.tips/blog/incremental-backup-linux/</url></story><parent_chain><item><author>funkaster</author><text>I used to have a &amp;quot;poor man&amp;#x27;s time machine&amp;quot; system based on rsync + hard links to files that didn&amp;#x27;t change with new backups. Essentially it was the same concept than time machine. Of course you couldn&amp;#x27;t upload a single &amp;quot;snapshot&amp;quot; because tar wouldn&amp;#x27;t know what&amp;#x27;s a hard link. One advantage of using rsync is that you can also keep track of things you delete.&lt;p&gt;Today I&amp;#x27;m using zfs with real snapshots. For systems with no zfs support (my wife&amp;#x27;s iMac for instance), I have a zfs fs that those systems rsync to, after the rsync is done I create a snapshot. All scripted. The snapshots can be stored in another server for an additional layer of backup, or incrementally send them to s3 if you want.</text></item></parent_chain><comment><author>Rotareti</author><text>&lt;i&gt;&amp;gt; I used to have a &amp;quot;poor man&amp;#x27;s time machine&amp;quot; system based on rsync + hard links to files that didn&amp;#x27;t change with new backups.&lt;/i&gt;&lt;p&gt;I build something similar, that runs on a Raspberry Pi and creates backups for the machines in my home network [0]. The Pi pings each machine every hour, if a machine is online and a backup is due, it starts a backup process. My Pi uses a USB battery as a UPS (unlimited power supply) [1]. I put all the hardware in a little medicine cabinet on the wall [1]. It&amp;#x27;s been running stable for month now, without a single reboot. It needs about 15 minutes to backup my dev machine over WiFi. It&amp;#x27;s a little independent backup module, I&amp;#x27;m really happy with it. :)&lt;p&gt;[0] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;feluxe&amp;#x2F;very-hungry-pi&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;feluxe&amp;#x2F;very-hungry-pi&lt;/a&gt;&lt;p&gt;[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.amazon.de&amp;#x2F;gp&amp;#x2F;product&amp;#x2F;B00FAU7ZB2&amp;#x2F;ref=oh_aui_search_detailpage?ie=UTF8&amp;amp;psc=1&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.amazon.de&amp;#x2F;gp&amp;#x2F;product&amp;#x2F;B00FAU7ZB2&amp;#x2F;ref=oh_aui_searc...&lt;/a&gt;&lt;p&gt;[2] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.amazon.de&amp;#x2F;HMF-Medizinschrank-Arzneischrank-Hausapotheke-Original&amp;#x2F;dp&amp;#x2F;B002CVKEWY&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.amazon.de&amp;#x2F;HMF-Medizinschrank-Arzneischrank-Hausa...&lt;/a&gt;</text></comment>
<story><title>Incremental Backups Using GNU Tar and S3</title><url>https://ops.tips/blog/incremental-backup-linux/</url></story><parent_chain><item><author>funkaster</author><text>I used to have a &amp;quot;poor man&amp;#x27;s time machine&amp;quot; system based on rsync + hard links to files that didn&amp;#x27;t change with new backups. Essentially it was the same concept than time machine. Of course you couldn&amp;#x27;t upload a single &amp;quot;snapshot&amp;quot; because tar wouldn&amp;#x27;t know what&amp;#x27;s a hard link. One advantage of using rsync is that you can also keep track of things you delete.&lt;p&gt;Today I&amp;#x27;m using zfs with real snapshots. For systems with no zfs support (my wife&amp;#x27;s iMac for instance), I have a zfs fs that those systems rsync to, after the rsync is done I create a snapshot. All scripted. The snapshots can be stored in another server for an additional layer of backup, or incrementally send them to s3 if you want.</text></item></parent_chain><comment><author>derekp7</author><text>I wrote a tool called Snebu (www.snebu.com), which is intended to give you the same thing as rsync + hard links.  Uses find and tar to grab file metadata and retrieve files, on the back end it stores file data in .lzo compressed files with all metadata for each snapshot (including symlinks, hardlinks, etc) stored in a sqlite3 DB file.&lt;p&gt;The design goals were to make something that didn&amp;#x27;t store data in a proprietary format (you can analyse the backups using straight sql commands, and access the data using lzop), and to be able to back up systems without installing a client agent on them, and support compression, and avoid filesystem issues you run into with a large number of hard links (such as &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=8305283&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=8305283&lt;/a&gt;).  So far I&amp;#x27;ve been using it to back up a few dozen RHEL 4 - RHEL 7 systems over the last couple years without issues.</text></comment>