<story><title>Old box, dumb code, few thousand connections, no big deal</title><url>https://rachelbythebay.com/w/2020/05/07/serv/</url></story><parent_chain><item><author>highfrequency</author><text>&amp;gt; single-threaded nonvectorized C wastes on the order of 97% of your computer&amp;#x27;s computational power&lt;p&gt;Can you elaborate on what this means exactly? For example, is there some reasonable C code that runs 33 times slower than some other ideal code? In what sense are we wasting 97% of our computer&amp;#x27;s computational power?</text></item><item><author>kragen</author><text>Rachel presumably wrote her server in a reasonable language like C++ (though I don&amp;#x27;t see a link to her source), but when I wrote httpdito⁰ ¹ ² I wrote it in assembly, and it can handle 2048 concurrent connections on similarly outdated hardware despite spawning an OS &lt;i&gt;process&lt;/i&gt; per connection, more than one concurrent connection per byte of executable†.  (It could handle more, but I had to set a limit somewhere.)  It just serves files from the filesystem.  It of course doesn&amp;#x27;t use epoll, but maybe it should — instead of Rachel&amp;#x27;s 50k requests per second, it can only handle about 20k or 30k on my old laptop.  IIRC I wrote it in one night.&lt;p&gt;It might sound like I&amp;#x27;m trying to steal her thunder, but mostly what I&amp;#x27;m trying to say is &lt;i&gt;she is right.  Listen to her.  Here is further evidence that she is right.&lt;/i&gt;&lt;p&gt;As I wrote in &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;gitlab.com&amp;#x2F;kragen&amp;#x2F;derctuo&amp;#x2F;blob&amp;#x2F;master&amp;#x2F;vector-vm.md&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;gitlab.com&amp;#x2F;kragen&amp;#x2F;derctuo&amp;#x2F;blob&amp;#x2F;master&amp;#x2F;vector-vm.md&lt;/a&gt;, single-threaded nonvectorized C wastes on the order of 97% of your computer&amp;#x27;s computational power, and typical interpreted languages like Python waste about 99.9% of it.  There&amp;#x27;s a huge amount of potential that&amp;#x27;s going untapped.&lt;p&gt;I feel like with modern technologies like LuaJIT, LevelDB, ØMQ, FlatBuffers, ISPC, seL4, and of course modern Linux, we ought to be able to do a lot of things that we couldn&amp;#x27;t even imagine doing in 2005, because they would have been far too inefficient.  But our imaginations are still too limited, and industry is not doing a very good job of imagining things.&lt;p&gt;—&lt;p&gt;⁰ &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;canonical.org&amp;#x2F;~kragen&amp;#x2F;sw&amp;#x2F;dev3&amp;#x2F;server.s&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;canonical.org&amp;#x2F;~kragen&amp;#x2F;sw&amp;#x2F;dev3&amp;#x2F;server.s&lt;/a&gt;&lt;p&gt;¹ &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;canonical.org&amp;#x2F;~kragen&amp;#x2F;sw&amp;#x2F;dev3&amp;#x2F;httpdito-readme&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;canonical.org&amp;#x2F;~kragen&amp;#x2F;sw&amp;#x2F;dev3&amp;#x2F;httpdito-readme&lt;/a&gt;&lt;p&gt;² &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=6908064&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=6908064&lt;/a&gt;&lt;p&gt;† It&amp;#x27;s actually bloated up to 2060 bytes now because I added PDF and CSS content-types to it, but you can git clone the .git subdirectory and check out the older versions that were under 2000 bytes.</text></item></parent_chain><comment><author>augustt</author><text>A good example of getting a ~3000x speedup from naive matrix multiplication in C here (slides 20 onward): &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;ocw.mit.edu&amp;#x2F;courses&amp;#x2F;electrical-engineering-and-computer-science&amp;#x2F;6-172-performance-engineering-of-software-systems-fall-2018&amp;#x2F;lecture-slides&amp;#x2F;MIT6_172F18_lec1.pdf&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;ocw.mit.edu&amp;#x2F;courses&amp;#x2F;electrical-engineering-and-compu...&lt;/a&gt;&lt;p&gt;Includes a 9-level nested for loop, which is always great to see.</text></comment>
<story><title>Old box, dumb code, few thousand connections, no big deal</title><url>https://rachelbythebay.com/w/2020/05/07/serv/</url></story><parent_chain><item><author>highfrequency</author><text>&amp;gt; single-threaded nonvectorized C wastes on the order of 97% of your computer&amp;#x27;s computational power&lt;p&gt;Can you elaborate on what this means exactly? For example, is there some reasonable C code that runs 33 times slower than some other ideal code? In what sense are we wasting 97% of our computer&amp;#x27;s computational power?</text></item><item><author>kragen</author><text>Rachel presumably wrote her server in a reasonable language like C++ (though I don&amp;#x27;t see a link to her source), but when I wrote httpdito⁰ ¹ ² I wrote it in assembly, and it can handle 2048 concurrent connections on similarly outdated hardware despite spawning an OS &lt;i&gt;process&lt;/i&gt; per connection, more than one concurrent connection per byte of executable†.  (It could handle more, but I had to set a limit somewhere.)  It just serves files from the filesystem.  It of course doesn&amp;#x27;t use epoll, but maybe it should — instead of Rachel&amp;#x27;s 50k requests per second, it can only handle about 20k or 30k on my old laptop.  IIRC I wrote it in one night.&lt;p&gt;It might sound like I&amp;#x27;m trying to steal her thunder, but mostly what I&amp;#x27;m trying to say is &lt;i&gt;she is right.  Listen to her.  Here is further evidence that she is right.&lt;/i&gt;&lt;p&gt;As I wrote in &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;gitlab.com&amp;#x2F;kragen&amp;#x2F;derctuo&amp;#x2F;blob&amp;#x2F;master&amp;#x2F;vector-vm.md&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;gitlab.com&amp;#x2F;kragen&amp;#x2F;derctuo&amp;#x2F;blob&amp;#x2F;master&amp;#x2F;vector-vm.md&lt;/a&gt;, single-threaded nonvectorized C wastes on the order of 97% of your computer&amp;#x27;s computational power, and typical interpreted languages like Python waste about 99.9% of it.  There&amp;#x27;s a huge amount of potential that&amp;#x27;s going untapped.&lt;p&gt;I feel like with modern technologies like LuaJIT, LevelDB, ØMQ, FlatBuffers, ISPC, seL4, and of course modern Linux, we ought to be able to do a lot of things that we couldn&amp;#x27;t even imagine doing in 2005, because they would have been far too inefficient.  But our imaginations are still too limited, and industry is not doing a very good job of imagining things.&lt;p&gt;—&lt;p&gt;⁰ &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;canonical.org&amp;#x2F;~kragen&amp;#x2F;sw&amp;#x2F;dev3&amp;#x2F;server.s&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;canonical.org&amp;#x2F;~kragen&amp;#x2F;sw&amp;#x2F;dev3&amp;#x2F;server.s&lt;/a&gt;&lt;p&gt;¹ &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;canonical.org&amp;#x2F;~kragen&amp;#x2F;sw&amp;#x2F;dev3&amp;#x2F;httpdito-readme&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;canonical.org&amp;#x2F;~kragen&amp;#x2F;sw&amp;#x2F;dev3&amp;#x2F;httpdito-readme&lt;/a&gt;&lt;p&gt;² &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=6908064&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=6908064&lt;/a&gt;&lt;p&gt;† It&amp;#x27;s actually bloated up to 2060 bytes now because I added PDF and CSS content-types to it, but you can git clone the .git subdirectory and check out the older versions that were under 2000 bytes.</text></item></parent_chain><comment><author>umvi</author><text>I think it&amp;#x27;s referring to simd instructions [0].&lt;p&gt;I&amp;#x27;m still kind of a newb myself but from what I understand these are special CPU instructions that allow you execute the same instruction in parallel against multiple data points. This allows you to eke out a lot more performance. It&amp;#x27;s how simdjson[1] is able to outperform all other C++ json parsers.&lt;p&gt;[0] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;SIMD&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;SIMD&lt;/a&gt;&lt;p&gt;[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;simdjson&amp;#x2F;simdjson&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;simdjson&amp;#x2F;simdjson&lt;/a&gt;</text></comment>