<story><title>The End of CI</title><url>https://matt-rickard.com/the-end-of-ci/</url></story><parent_chain><item><author>Frost1x</author><text>I always found it interesting how git was adopted by industry where a large part of its complexity and feature set is built around suopporting fully distributed developer, developers who could lack internet access for months at a time but need to work collaboratively in a team. Then, industry took that distributed approach and strapped project and team management systems with web based front ends like Github, GitLab, etc. around git and created CI&amp;#x2F;CD pipelines that built ontop of or in parallel to those environments--all of which were centralized. While it&amp;#x27;s of course still true that your underlying repo instances still give you a lot of flexibility away from those  centralized environments, the centralized environments themselves are critical in regular daily workflows, negating a lot of the flexibility git as a VCS adds.&lt;p&gt;In large part, industry could largely use a significantly simpler model than git for VCS since many features are largely moot on a daily basis. Many could probably get away with SVN in large part, for example (although branching isn&amp;#x27;t nearly as good).&lt;p&gt;I think you raise a great point in that we need to look at how processes have evolved ontop of version control and look at adapting those to a similar model. In some cases it&amp;#x27;s just not practical because the way infrastructure testing an deployment works but it&amp;#x27;s the direction I think we should be going. In fact, the first step would be to make issue systems and project management interfaces provided by efforts like GitLab and GitHub available in a local capable, distributed fashion. Clearly communications often require some degree of centralization or at least peer message propagation but there&amp;#x27;s no reason that information can&amp;#x27;t be separated from the infrastructure that displays and interacts with it.</text></item><item><author>lukeasrodgers</author><text>What if it was possible to have a local build system that would run tests etc and provide a cryptographically secure hash proving that the tests passed, the build was good, etc then a “centralized” CI system just verified that before permitting merge&amp;#x2F;deploy.
There’s a lot of hand waving there but it doesn’t seem wildly implausible. 
Local use of containers or something like nixos could provide sufficient protection against the “works on my machine but not in production” problem.</text></item><item><author>hinkley</author><text>I always have to be careful responding to things like this because I get sucked into the trap of thinking “we” means myself, and it just needs to mean “most of us” for whatever they’re talking about to be worrisome or even true.&lt;p&gt;I do worry that a lot of people are missing the first principles where CI is concerned, but a few of us still remember. Whether that’s enough is always a question.&lt;p&gt;&amp;gt; First, CI rarely knows what has changed, so everything needs to be rebuilt and retested&lt;p&gt;CI is the impartial judge in your system. That is the ultimate goal and much of the value. Don’t believe &lt;i&gt;me&lt;/i&gt; when I say that your check in broke something. Believe the robot. It does not think, it does not interpret errors (which is why intermittent failures poison the well). It just knows that whatever you did was not good.&lt;p&gt;We make it stupid simple because the more ways you try to interpret things, the higher the likelihood the system will either miss an error or report imaginary ones. And in this case I am very sorry Mr Lasso, but it is in fact the hope that kills. Optimism makes us try things, things others would give up on. That’s what makes a piece of software good. Better. But too much and you start ignoring very real problems and make optimism into a liability. I’ve seen it over and over again, take a build with intermittent failures and people will check in broken code and assume it’s the intermittent problem and not their change.&lt;p&gt;Ideally, when you start breaking builds you’ve found your limits. But the build has to break when I break the code, and it needs to do it repeatably, because if I can’t reproduce the error I risk telling myself stories, and foisting off debugging to other people, which is a big no no.</text></item></parent_chain><comment><author>morelisp</author><text>&amp;gt; Then, industry took that distributed approach and strapped project and team management systems with web based front ends like Github, GitLab, etc. around git and created CI&amp;#x2F;CD pipelines that built ontop of or in parallel to those environments--all of which were centralized.&lt;p&gt;A different view on this is that my CI system is a kind of colleague, responsible for a lot of what testers, ops, and build eng people toiled at before. It is another aspect of the decentralized system where a robot can also check out and work with the code even while humans continue to write more, which is considerably more painful under e.g. the SVN model. My human colleagues and I still share plenty of code directly, via git pulls and other tools.</text></comment>
<story><title>The End of CI</title><url>https://matt-rickard.com/the-end-of-ci/</url></story><parent_chain><item><author>Frost1x</author><text>I always found it interesting how git was adopted by industry where a large part of its complexity and feature set is built around suopporting fully distributed developer, developers who could lack internet access for months at a time but need to work collaboratively in a team. Then, industry took that distributed approach and strapped project and team management systems with web based front ends like Github, GitLab, etc. around git and created CI&amp;#x2F;CD pipelines that built ontop of or in parallel to those environments--all of which were centralized. While it&amp;#x27;s of course still true that your underlying repo instances still give you a lot of flexibility away from those  centralized environments, the centralized environments themselves are critical in regular daily workflows, negating a lot of the flexibility git as a VCS adds.&lt;p&gt;In large part, industry could largely use a significantly simpler model than git for VCS since many features are largely moot on a daily basis. Many could probably get away with SVN in large part, for example (although branching isn&amp;#x27;t nearly as good).&lt;p&gt;I think you raise a great point in that we need to look at how processes have evolved ontop of version control and look at adapting those to a similar model. In some cases it&amp;#x27;s just not practical because the way infrastructure testing an deployment works but it&amp;#x27;s the direction I think we should be going. In fact, the first step would be to make issue systems and project management interfaces provided by efforts like GitLab and GitHub available in a local capable, distributed fashion. Clearly communications often require some degree of centralization or at least peer message propagation but there&amp;#x27;s no reason that information can&amp;#x27;t be separated from the infrastructure that displays and interacts with it.</text></item><item><author>lukeasrodgers</author><text>What if it was possible to have a local build system that would run tests etc and provide a cryptographically secure hash proving that the tests passed, the build was good, etc then a “centralized” CI system just verified that before permitting merge&amp;#x2F;deploy.
There’s a lot of hand waving there but it doesn’t seem wildly implausible. 
Local use of containers or something like nixos could provide sufficient protection against the “works on my machine but not in production” problem.</text></item><item><author>hinkley</author><text>I always have to be careful responding to things like this because I get sucked into the trap of thinking “we” means myself, and it just needs to mean “most of us” for whatever they’re talking about to be worrisome or even true.&lt;p&gt;I do worry that a lot of people are missing the first principles where CI is concerned, but a few of us still remember. Whether that’s enough is always a question.&lt;p&gt;&amp;gt; First, CI rarely knows what has changed, so everything needs to be rebuilt and retested&lt;p&gt;CI is the impartial judge in your system. That is the ultimate goal and much of the value. Don’t believe &lt;i&gt;me&lt;/i&gt; when I say that your check in broke something. Believe the robot. It does not think, it does not interpret errors (which is why intermittent failures poison the well). It just knows that whatever you did was not good.&lt;p&gt;We make it stupid simple because the more ways you try to interpret things, the higher the likelihood the system will either miss an error or report imaginary ones. And in this case I am very sorry Mr Lasso, but it is in fact the hope that kills. Optimism makes us try things, things others would give up on. That’s what makes a piece of software good. Better. But too much and you start ignoring very real problems and make optimism into a liability. I’ve seen it over and over again, take a build with intermittent failures and people will check in broken code and assume it’s the intermittent problem and not their change.&lt;p&gt;Ideally, when you start breaking builds you’ve found your limits. But the build has to break when I break the code, and it needs to do it repeatably, because if I can’t reproduce the error I risk telling myself stories, and foisting off debugging to other people, which is a big no no.</text></item></parent_chain><comment><author>Silhouette</author><text>It&amp;#x27;s a recurring and perhaps inevitable problem. We&amp;#x27;ve collectively been building software for a long time but any given individual hasn&amp;#x27;t. No-one has seen all the good ideas that have gone before and sometimes been forgotten. Everyone is influenced to some degree by whatever ideas are currently popular within their group.&lt;p&gt;We developed distributed VCSes and the ability to do smart things with branches and combining contributions from people all over the world at different times. Then we somehow still ended up with centralised processes where everyone has to merge to trunk frequently and if GitHub goes down then the world stops turning. It turns out that just using git as git wasn&amp;#x27;t such a bad idea after all.&lt;p&gt;We were told that dynamic languages like JS and Python are so much more productive and that this was essential for fast-moving startups to be competitive. Today the industry is moving sharply from JS to TypeScript and even Python now has similar tools. It turns out that static typing was better for building robust software at scale than relying entirely on unit tests after all.&lt;p&gt;Just wait until lots of developers who have only ever worked with JS or Python learn what a compiled executable is. If you tell them that xcopy deployment even to a huge farm of web servers is no big deal when all you&amp;#x27;re copying is one executable file and maybe a .env with the secrets, it&amp;#x27;ll blow their minds...</text></comment>