<story><title>Use one big server</title><url>https://specbranch.com/posts/one-big-server/</url></story><parent_chain><item><author>thayne</author><text>One issue I&amp;#x27;ve seen with this is that if you have a single, very large database, it can take a very, very long time to restore from backups. Or for that matter just taking backups.&lt;p&gt;I&amp;#x27;d be interested to know if anyone has a good solution for that.</text></item><item><author>belak</author><text>This is absolutely true - when I was at Bitbucket (ages ago at this point) and we were having issues with our DB server (mostly due to scaling), almost everyone we talked to said &amp;quot;buy a bigger box until you can&amp;#x27;t any more&amp;quot; because of how complex (and indirectly expensive) the alternatives are - sharding and microservices both have a ton more failure points than a single large box.&lt;p&gt;I&amp;#x27;m sure they eventually moved off that single primary box, but for many years Bitbucket was run off 1 primary in each datacenter (with a failover), and a few read-only copies. If you&amp;#x27;re getting to the point where one database isn&amp;#x27;t enough, you&amp;#x27;re either doing something pretty weird, are working on a specific problem which needs a more complicated setup, or have grown to the point where investing in a microservice architecture starts to make sense.</text></item><item><author>cfors</author><text>Yep, there&amp;#x27;s a premium on making your architecture more cloudy. However, the best point for Use One Big Server is not necessarily running your big monolithic API server, but your database.&lt;p&gt;Use One Big Database.&lt;p&gt;Seriously. If you are a backend engineer, nothing is worse than breaking up your data into self contained service databases, where everything is passed over Rest&amp;#x2F;RPC. Your product asks will consistently want to combine these data sources (they don&amp;#x27;t know how your distributed databases look, and oftentimes they really do not care).&lt;p&gt;It is so much easier to do these joins efficiently in a single database than fanning out RPC calls to multiple different databases, not to mention dealing with inconsistencies, lack of atomicity, etc. etc. Spin up a specific reader of that database if there needs to be OLAP queries, or use a message bus. But keep your OLTP data within one database for as long as possible.&lt;p&gt;You can break apart a stateless microservice, but there are few things as stagnant in the world of software than data. It will keep you nimble for new product features. The boxes that they offer on cloud vendors today for managed databases are giant!</text></item></parent_chain><comment><author>dsr_</author><text>Here&amp;#x27;s the way it works for, say, Postgresql:&lt;p&gt;- you rsync or zfs send the database files from machine A to machine B. You would like the database to be off during this process, which will make it consistent. The big advantage of ZFS is that you can stop PG, snapshot the filesystem, and turn PG on again immediately, then send the snapshot. Machine B is now a cold backup replica of A. Your loss potential is limited to the time between backups.&lt;p&gt;- after the previous step is completed, you arrange for machine A to send WAL files to machine B. It&amp;#x27;s well documented. You could use rsync or scp here. It happens automatically and frequently. Machine B is now a warm replica of A -- if you need to turn it on in an emergency, you will only have lost one WAL file&amp;#x27;s worth of changes.&lt;p&gt;- after that step is completed, you give machine B credentials to login to A for live replication. Machine B is now a live, very slightly delayed read-only replica of A. Anything that A processes will be updated on B as soon as it is received.&lt;p&gt;You can go further and arrange to load balance requests between read-only replicas, while sending the write requests to the primary; you can look at Citus (now open source) to add multi-primary clustering.</text></comment>
<story><title>Use one big server</title><url>https://specbranch.com/posts/one-big-server/</url></story><parent_chain><item><author>thayne</author><text>One issue I&amp;#x27;ve seen with this is that if you have a single, very large database, it can take a very, very long time to restore from backups. Or for that matter just taking backups.&lt;p&gt;I&amp;#x27;d be interested to know if anyone has a good solution for that.</text></item><item><author>belak</author><text>This is absolutely true - when I was at Bitbucket (ages ago at this point) and we were having issues with our DB server (mostly due to scaling), almost everyone we talked to said &amp;quot;buy a bigger box until you can&amp;#x27;t any more&amp;quot; because of how complex (and indirectly expensive) the alternatives are - sharding and microservices both have a ton more failure points than a single large box.&lt;p&gt;I&amp;#x27;m sure they eventually moved off that single primary box, but for many years Bitbucket was run off 1 primary in each datacenter (with a failover), and a few read-only copies. If you&amp;#x27;re getting to the point where one database isn&amp;#x27;t enough, you&amp;#x27;re either doing something pretty weird, are working on a specific problem which needs a more complicated setup, or have grown to the point where investing in a microservice architecture starts to make sense.</text></item><item><author>cfors</author><text>Yep, there&amp;#x27;s a premium on making your architecture more cloudy. However, the best point for Use One Big Server is not necessarily running your big monolithic API server, but your database.&lt;p&gt;Use One Big Database.&lt;p&gt;Seriously. If you are a backend engineer, nothing is worse than breaking up your data into self contained service databases, where everything is passed over Rest&amp;#x2F;RPC. Your product asks will consistently want to combine these data sources (they don&amp;#x27;t know how your distributed databases look, and oftentimes they really do not care).&lt;p&gt;It is so much easier to do these joins efficiently in a single database than fanning out RPC calls to multiple different databases, not to mention dealing with inconsistencies, lack of atomicity, etc. etc. Spin up a specific reader of that database if there needs to be OLAP queries, or use a message bus. But keep your OLTP data within one database for as long as possible.&lt;p&gt;You can break apart a stateless microservice, but there are few things as stagnant in the world of software than data. It will keep you nimble for new product features. The boxes that they offer on cloud vendors today for managed databases are giant!</text></item></parent_chain><comment><author>mike_hearn</author><text>Presumably it doesn&amp;#x27;t matter if you break your DB up into smaller DBs, you still have the same amount of data to back up no matter what. However, now you also have the problem of snapshot consistency to worry about.&lt;p&gt;If you need to backup&amp;#x2F;restore just one set of tables, you can do that with a single DB server without taking the rest offline.</text></comment>