<story><title>Write Gitlab CI Pipelines in Python Code</title><url>https://gitlab.com/dbsystel/gitlab-ci-python-library</url></story><parent_chain><item><author>octopoc</author><text>What I would really like to see is a CI system that lets me write a script in a language of my choice instead of defining a pipeline config file. That way I can run the pipeline locally, put breakpoints in, etc.&lt;p&gt;Nuke [1] gets close but there are still a lot of tasks that don&amp;#x27;t have C# bindings, such as publishing build artifacts and uploading test results.&lt;p&gt;While I&amp;#x27;m dreaming about my perfect CI, I&amp;#x27;d also like the ability to download benchmark results from previous commits so that I can generate trend graphs and publish them in the build results.&lt;p&gt;To do this right the CI system would have to have an API using REST, GraphQL, gRPC or some such API format that generates clients in many languages. That way they don&amp;#x27;t have to maintain bindings in every language.&lt;p&gt;[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;nuke.build&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;nuke.build&amp;#x2F;&lt;/a&gt;</text></item></parent_chain><comment><author>AlphaSite</author><text>I think this is more of an issue with people misusing CI systems, your gitlab-CI&amp;#x2F;Jenkins&amp;#x2F;concourse&amp;#x2F;etc shouldn’t have any real logic in it, it’s just glue code.&lt;p&gt;Bazel&amp;#x2F;Gradle&amp;#x2F;Maven&amp;#x2F;etc tend push you in the right direction, bash, etc dont.&lt;p&gt;Or to put it succinctly, you need both a build system and a CI system, they are not the same thing.</text></comment>
<story><title>Write Gitlab CI Pipelines in Python Code</title><url>https://gitlab.com/dbsystel/gitlab-ci-python-library</url></story><parent_chain><item><author>octopoc</author><text>What I would really like to see is a CI system that lets me write a script in a language of my choice instead of defining a pipeline config file. That way I can run the pipeline locally, put breakpoints in, etc.&lt;p&gt;Nuke [1] gets close but there are still a lot of tasks that don&amp;#x27;t have C# bindings, such as publishing build artifacts and uploading test results.&lt;p&gt;While I&amp;#x27;m dreaming about my perfect CI, I&amp;#x27;d also like the ability to download benchmark results from previous commits so that I can generate trend graphs and publish them in the build results.&lt;p&gt;To do this right the CI system would have to have an API using REST, GraphQL, gRPC or some such API format that generates clients in many languages. That way they don&amp;#x27;t have to maintain bindings in every language.&lt;p&gt;[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;nuke.build&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;nuke.build&amp;#x2F;&lt;/a&gt;</text></item></parent_chain><comment><author>qbasic_forever</author><text>Check out tekton CI (&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;tekton.dev&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;tekton.dev&amp;#x2F;&lt;/a&gt;), it&amp;#x27;s a Kubernetes operator to run a CI pipeline that&amp;#x27;s defined as commands running inside any container.  Use any language, any commands, etc--as long as you can get a container image, you&amp;#x27;re good to go.  There&amp;#x27;s a growing set of community created and curated actions to do common things too: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;tektoncd&amp;#x2F;catalog&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;tektoncd&amp;#x2F;catalog&lt;/a&gt;&lt;p&gt;Yeah you need a k8s cluster, but even a simple kind dev cluster that you spin up in 30 seconds with one command on your laptop will work.&lt;p&gt;I like it a lot because it enforces very little structure on you and doesn&amp;#x27;t reinvent everything.  Stuff like storage (either ephermeral or existing volumes), secrets, configuration, etc. are already modeled and supported by Kubernetes and tekton can use all of that natively.  And since it&amp;#x27;s all k8s native stuff you have all of the power of k8s, like its entire API for manipulating and managing execution, exposing services, etc.  There is very little cognitive overhead or new things to learn once you know k8s.&lt;p&gt;If you&amp;#x27;re really averse to k8s though, check out drone.  It has a local execution mode that is similar and just runs whatever pipeline commands you want in docker containers. &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;drone&amp;#x2F;drone&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;drone&amp;#x2F;drone&lt;/a&gt;  Batect is another even more minimal tool that&amp;#x27;s effectively just a docker-based workflow system: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;batect&amp;#x2F;batect&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;batect&amp;#x2F;batect&lt;/a&gt;</text></comment>