<story><title>GitHub&apos;s Unicorn Setup</title><url>http://github.com/blog/517-unicorn</url></story><parent_chain><item><author>antonovka</author><text>The argument encompassed two aspects of the model:&lt;p&gt;1) &quot;Threads are out -- processes are better than threads.&quot;&lt;p&gt;2) Process-per-connection architectures.&lt;p&gt;The first is demonstrably false -- for instance, look at Erlang, which maps lightweight erlang processes to operating system threads, providing SMP scalability at a low cost without running into Github&apos;s issues with mongrel &quot;thread-killing&quot;. More broadly used, look at Servlets and the Servlet 3.0 support for async comet-style event-based request handling. Each request is handled on a thread as necessary. Inter-thread communication (where necessary) is cheap, and this scales just fine.&lt;p&gt;If you use the fork() model and a long-running request blocks the entire process, comet is basically a non-starter. This is why people are interested (and implement) lightweight threads, coroutines, and restartable request implementations.&lt;p&gt;For a conceptual challenge, consider how you would implement a live web chat system that can scale up to a considerable number of clients, with extremely low resource usage, and instant message distribution (no polling). Locally, we implemented this with async servlet support and M:N scheduled scala actors -- a blocking HTTP request doesn&apos;t hold a thread or process hostage in the web server or the application, and we can scale up to enormous number of live clients on one machine.&lt;p&gt;The second was merely a lack of understanding of the model. In implementations where fork() is used as an alternative to   threads and multiple connections are not handled per sub-process, you quickly run into scaling issues with subprocess memory utilization. In cases where subprocesses handle multiple connections via an event mechanism, you&apos;re just using fork() instead of threads.&lt;p&gt;I&apos;m wondering how many servers github is using to keep up with load -- If each 8 core 16 gigabyte (!!!) server can actually only handle 16 concurrent requests via a pool of 16 workers, that&apos;s an incredibly poor (and expensive) scaling model.</text></item><item><author>aaronblohowiak</author><text>Right, but it is being handled by processes and not threads, which the argument was about the other day.</text></item><item><author>seiji</author><text>It&apos;s connection pooling, not a fork-per-accept web server.  Each worker is select/epoll/kqueue&apos;ing for individual requests.</text></item><item><author>dschobel</author><text>&lt;i&gt;When the Unicorn master starts, it loads our app into memory. As soon as it’s ready to serve requests it forks 16 workers. Those workers then select() on the socket, only serving requests they’re capable of handling. In this way the kernel handles the load balancing for us.&lt;/i&gt;&lt;p&gt;Wasn&apos;t there a heated debate here just the other day about the prefork model?&lt;p&gt;Guess it&apos;s at least back en vogue @github.</text></item></parent_chain><comment><author>mojombo</author><text>Each box can handle plenty more than 16 workers, but we currently don&apos;t need that many, so it&apos;s pointless to run them. Our frontend machines generally sit about 60% idle. We wanted to have plenty of headroom on our new setup.&lt;p&gt;Also, our Ruby application code spends very little time blocking on external resources (compared to time spent in Ruby execution), so having async execution on these wouldn&apos;t give us that much more efficiency. It&apos;s true that running Ruby code is generally more expensive (dollar-wise) per given task than other languages, but we made a conscious decision to make that tradeoff for the productivity gains that Ruby affords us. We use Erlang and EventMachine in other pieces of the architecture where they make sense.&lt;p&gt;The simple fact is that Unicorn works better for us (and our specific use case) than anything else we&apos;ve tried, and so we&apos;re using it.</text></comment>
<story><title>GitHub&apos;s Unicorn Setup</title><url>http://github.com/blog/517-unicorn</url></story><parent_chain><item><author>antonovka</author><text>The argument encompassed two aspects of the model:&lt;p&gt;1) &quot;Threads are out -- processes are better than threads.&quot;&lt;p&gt;2) Process-per-connection architectures.&lt;p&gt;The first is demonstrably false -- for instance, look at Erlang, which maps lightweight erlang processes to operating system threads, providing SMP scalability at a low cost without running into Github&apos;s issues with mongrel &quot;thread-killing&quot;. More broadly used, look at Servlets and the Servlet 3.0 support for async comet-style event-based request handling. Each request is handled on a thread as necessary. Inter-thread communication (where necessary) is cheap, and this scales just fine.&lt;p&gt;If you use the fork() model and a long-running request blocks the entire process, comet is basically a non-starter. This is why people are interested (and implement) lightweight threads, coroutines, and restartable request implementations.&lt;p&gt;For a conceptual challenge, consider how you would implement a live web chat system that can scale up to a considerable number of clients, with extremely low resource usage, and instant message distribution (no polling). Locally, we implemented this with async servlet support and M:N scheduled scala actors -- a blocking HTTP request doesn&apos;t hold a thread or process hostage in the web server or the application, and we can scale up to enormous number of live clients on one machine.&lt;p&gt;The second was merely a lack of understanding of the model. In implementations where fork() is used as an alternative to   threads and multiple connections are not handled per sub-process, you quickly run into scaling issues with subprocess memory utilization. In cases where subprocesses handle multiple connections via an event mechanism, you&apos;re just using fork() instead of threads.&lt;p&gt;I&apos;m wondering how many servers github is using to keep up with load -- If each 8 core 16 gigabyte (!!!) server can actually only handle 16 concurrent requests via a pool of 16 workers, that&apos;s an incredibly poor (and expensive) scaling model.</text></item><item><author>aaronblohowiak</author><text>Right, but it is being handled by processes and not threads, which the argument was about the other day.</text></item><item><author>seiji</author><text>It&apos;s connection pooling, not a fork-per-accept web server.  Each worker is select/epoll/kqueue&apos;ing for individual requests.</text></item><item><author>dschobel</author><text>&lt;i&gt;When the Unicorn master starts, it loads our app into memory. As soon as it’s ready to serve requests it forks 16 workers. Those workers then select() on the socket, only serving requests they’re capable of handling. In this way the kernel handles the load balancing for us.&lt;/i&gt;&lt;p&gt;Wasn&apos;t there a heated debate here just the other day about the prefork model?&lt;p&gt;Guess it&apos;s at least back en vogue @github.</text></item></parent_chain><comment><author>mrshoe</author><text>Where were all you people in this thread the other day when I seemed to be the only one who understood the other side of this debate?!? :-P&lt;p&gt;I&apos;m also very curious about the 16 workers per box model. Frankly, that github would ditch haproxy in favor of a pre-fork Ruby web server is shocking.&lt;p&gt;I&apos;m just glad we have CherryPy in the Python world.</text></comment>