<story><title>Notes on Distributed Systems for Young Bloods (2013)</title><url>https://www.somethingsimilar.com/2013/01/14/notes-on-distributed-systems-for-young-bloods/</url></story><parent_chain><item><author>Animats</author><text>It&amp;#x27;s a step backwards that distribution is being dealt with at the application level. Tandem Computers [1] had distribution at the OS and database level working well in the 1980s.  The technology is still available from HP.&lt;p&gt;Tandem&amp;#x27;s NonStop OS is fundamentally different from UNIX-like OSs. UNIX is a time-sharing system with a file system. NonStop is a transaction system with a database system. There&amp;#x27;s very little state outside the database. 
Programs are started, do one transaction, and exit. If there&amp;#x27;s a problem during a transaction, the entire transaction aborts and does not commit.&lt;p&gt;Tandem was running an average system uptime of about 10 years across their installed base, with the main cause of failure being human errors during hardware maintenance. One of their people wrote a paper on how to get the uptime up to 50 years.  Tandem&amp;#x27;s own hardware had heavy self-checking. The goal was to detect errors and abort, not fix them. Failing a transaction would result in a fallover to the paired backup machine, which would result in a retry and recovery.&lt;p&gt;Tandem&amp;#x27;s main problem was high hardware cost; you needed several of anything.  Also, because of a series of acquisitions, the system had to be ported from Tandem to MIPS to Alpha to HP-RISC to Itanium to, finally, x86. X86 lacks the checking hardware, so it&amp;#x27;s not as good for this.&lt;p&gt;[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Tandem_Computers&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Tandem_Computers&lt;/a&gt;</text></item></parent_chain><comment><author>nostrademons</author><text>The tricky thing about that is that most highly-reliable distributed systems these days evolve from startups that are not-at-all-reliable.&lt;p&gt;Tandem&amp;#x27;s technology is incredible, even by today&amp;#x27;s standards and certainly by the standards of 1974.  But they were designed for a world where you sold computers (hardware + software) to big established companies for millions of dollars.  Once you saturate that market (which Tandem, Stratus, DEC, IBM, etc. did by the late 80s), what&amp;#x27;s next?&lt;p&gt;Since the mid-90s, most tech innovation has been driven by creating &lt;i&gt;new&lt;/i&gt; companies, &lt;i&gt;new&lt;/i&gt; networks, and &lt;i&gt;new&lt;/i&gt; value chains to replace the old incumbents.  This process is inherently experimental, and requires that you start out with a small system that grows into a large system.  Buying a Tandem computer for the sharing-economy company you&amp;#x27;re starting out of your apartment is a non-starter; even if you could afford it, it couples you too tightly to a set of applications that probably isn&amp;#x27;t relevant for what you want to do.&lt;p&gt;That&amp;#x27;s why most of the interest in distributed systems today is in taking clutzy single-box programs that used to be Rails or Node apps and then turning them into robust, fault-tolerant systems that serve billions of users and never go down.  That&amp;#x27;s where the money is.  The market of existing big companies that will pay a lot for software is largely tapped-out; the upcoming growth is in working for small companies that are rapidly becoming big by replacing those incumbents.</text></comment>
<story><title>Notes on Distributed Systems for Young Bloods (2013)</title><url>https://www.somethingsimilar.com/2013/01/14/notes-on-distributed-systems-for-young-bloods/</url></story><parent_chain><item><author>Animats</author><text>It&amp;#x27;s a step backwards that distribution is being dealt with at the application level. Tandem Computers [1] had distribution at the OS and database level working well in the 1980s.  The technology is still available from HP.&lt;p&gt;Tandem&amp;#x27;s NonStop OS is fundamentally different from UNIX-like OSs. UNIX is a time-sharing system with a file system. NonStop is a transaction system with a database system. There&amp;#x27;s very little state outside the database. 
Programs are started, do one transaction, and exit. If there&amp;#x27;s a problem during a transaction, the entire transaction aborts and does not commit.&lt;p&gt;Tandem was running an average system uptime of about 10 years across their installed base, with the main cause of failure being human errors during hardware maintenance. One of their people wrote a paper on how to get the uptime up to 50 years.  Tandem&amp;#x27;s own hardware had heavy self-checking. The goal was to detect errors and abort, not fix them. Failing a transaction would result in a fallover to the paired backup machine, which would result in a retry and recovery.&lt;p&gt;Tandem&amp;#x27;s main problem was high hardware cost; you needed several of anything.  Also, because of a series of acquisitions, the system had to be ported from Tandem to MIPS to Alpha to HP-RISC to Itanium to, finally, x86. X86 lacks the checking hardware, so it&amp;#x27;s not as good for this.&lt;p&gt;[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Tandem_Computers&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Tandem_Computers&lt;/a&gt;</text></item></parent_chain><comment><author>GauntletWizard</author><text>That&amp;#x27;s not moving it to an OS level, that&amp;#x27;s rethinking everything to deal with the problem you think you have, but don&amp;#x27;t.&lt;p&gt;Making every subprocess transactional is a good idea, in theory. It makes it simple to clear and retry in case of failure. But by the time you&amp;#x27;ve gotten to that point, you&amp;#x27;re already halfway through the problem of distributed systems, dividing your program into small, transactional slices.&lt;p&gt;Once you&amp;#x27;re there, though, the Tandem architecture paints you into a corner. These database-oriented architectures do not scale to having large numbers of useful-sized transactions running at once. The bottlenecks of the OS and Database layer become insurmountable. Your architecture is a classic case of a local optimum.</text></comment>