<story><title>How a fix in Go 1.9 sped up our Gitaly service by 30x</title><url>https://about.gitlab.com/2018/01/23/how-a-fix-in-go-19-sped-up-our-gitaly-service-by-30x/</url></story><parent_chain><item><author>CapacitorSet</author><text>I wonder how much it would speed up if they were using libgit2 directly.</text></item><item><author>Animats</author><text>&lt;i&gt;Each Gitaly server instance was fork&amp;#x2F;exec&amp;#x27;ing Git processes about 20 times per second so we seemed to finally have a very promising lead.&lt;/i&gt;&lt;p&gt;What&amp;#x27;s really wrong here is that they&amp;#x27;re apparently spawning processes like crazy. Do they spawn a new process for each API call? That&amp;#x27;s like running CGI programs under Apache, like it&amp;#x27;s 1995.</text></item></parent_chain><comment><author>jacobvosmaer</author><text>I&amp;#x27;m on the Gitaly team.&lt;p&gt;As zegerjan wrote, Gitaly is a Go&amp;#x2F;Ruby hybrid.&lt;p&gt;The main Go process doesn&amp;#x27;t use libgit2 (for now) because we didn&amp;#x27;t want to have to deal with cgo. We already know how to deal with C extensions in Ruby, and we have a lot of existing Ruby application code that uses libgit2, so we still use it there. And that code works fine so I don&amp;#x27;t see us removing it.&lt;p&gt;In practice, sometimes spawning a Git process is faster than using libgit2, so why then not do that. Also for parts of our workload (handling Git push&amp;#x2F;pull operations), spawning a one-off process (git-upload-pack) is the most boring &amp;#x2F; tried-and-true approach.</text></comment>
<story><title>How a fix in Go 1.9 sped up our Gitaly service by 30x</title><url>https://about.gitlab.com/2018/01/23/how-a-fix-in-go-19-sped-up-our-gitaly-service-by-30x/</url></story><parent_chain><item><author>CapacitorSet</author><text>I wonder how much it would speed up if they were using libgit2 directly.</text></item><item><author>Animats</author><text>&lt;i&gt;Each Gitaly server instance was fork&amp;#x2F;exec&amp;#x27;ing Git processes about 20 times per second so we seemed to finally have a very promising lead.&lt;/i&gt;&lt;p&gt;What&amp;#x27;s really wrong here is that they&amp;#x27;re apparently spawning processes like crazy. Do they spawn a new process for each API call? That&amp;#x27;s like running CGI programs under Apache, like it&amp;#x27;s 1995.</text></item></parent_chain><comment><author>zegerjan</author><text>The Gitaly server has a Ruby component, but also a Go component. The Ruby server uses Rugged[1] and Gollum-lib[2] which both use libgit2.&lt;p&gt;The Go component doesn&amp;#x27;t have libgit2 binding yet, although we&amp;#x27;re looking into adding that later. That or maybe go-git[3]. But for now Gitaly is mainly focussed at migrating all git calls from the Rails monolith. Not introducing a new component now reduces the risks this project has.&lt;p&gt;[1]: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;libgit2&amp;#x2F;rugged&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;libgit2&amp;#x2F;rugged&amp;#x2F;&lt;/a&gt;
[2]: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;gitlab.slack.com&amp;#x2F;archives&amp;#x2F;C027N716H&amp;#x2F;p1516954304000263&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;gitlab.slack.com&amp;#x2F;archives&amp;#x2F;C027N716H&amp;#x2F;p151695430400026...&lt;/a&gt;
[3]: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;src-d&amp;#x2F;go-git&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;src-d&amp;#x2F;go-git&amp;#x2F;&lt;/a&gt;</text></comment>