<story><title>What&apos;s wrong with 1975 programming</title><url>http://varnish-cache.org/wiki/ArchitectNotes</url></story><parent_chain><item><author>tptacek</author><text>This is one of the best systems programming articles I&apos;ve read in a very long time. Short summary:&lt;p&gt;* Trust the VM system to figure out how to page things (hey, &apos;antirez, what&apos;s your take on that? You wrote an ad hoc pager for Redis.) instead of getting fancy, because if you get fancy you&apos;ll end up fighting with the VM system.&lt;p&gt;* Minimize memory accesses and minimize the likelihood that you&apos;ll compete with other cores for access to a cache line; for instance, instead of piecemeal allocations, make a large master allocation for a request and carve it out.&lt;p&gt;* Schedule threads in most-recently-busy order, so that when a thread goes to pick up a request it&apos;s maximally likely to have a pre-heated cache and resident set of variables to work with.</text></item></parent_chain><comment><author>antirez</author><text>If you have mostly big objects (relative to hardware page size) and the encoding in memory and when stored on disk are pretty similar, this is indeed a good idea (to trust the virtual memory I mean).&lt;p&gt;Otherwise... no way, if you have data structures everything is fragmented around (and you want to use a lot object sharing, caching, ... for performance, without to mention hash tables that are very cool at filling at least 1 byte of tons of pages even with 2% of data inside).&lt;p&gt;Also data structures can often be serialized on disk using 1/10 of the space.&lt;p&gt;Using the VM is cool, but not &lt;i&gt;so&lt;/i&gt; generally applicable. The proxy stuff is perfect. Also on-disk DB is perfect using the VM the other way around, to get a memory-cache for free, if you &lt;i&gt;don&apos;t&lt;/i&gt; need strict consistency (see MongoDB).</text></comment>
<story><title>What&apos;s wrong with 1975 programming</title><url>http://varnish-cache.org/wiki/ArchitectNotes</url></story><parent_chain><item><author>tptacek</author><text>This is one of the best systems programming articles I&apos;ve read in a very long time. Short summary:&lt;p&gt;* Trust the VM system to figure out how to page things (hey, &apos;antirez, what&apos;s your take on that? You wrote an ad hoc pager for Redis.) instead of getting fancy, because if you get fancy you&apos;ll end up fighting with the VM system.&lt;p&gt;* Minimize memory accesses and minimize the likelihood that you&apos;ll compete with other cores for access to a cache line; for instance, instead of piecemeal allocations, make a large master allocation for a request and carve it out.&lt;p&gt;* Schedule threads in most-recently-busy order, so that when a thread goes to pick up a request it&apos;s maximally likely to have a pre-heated cache and resident set of variables to work with.</text></item></parent_chain><comment><author>Locke1689</author><text>It&apos;s pretty good. Maybe because I&apos;m also a kernel developer, but these suggestions seem obvious to me. That is, instead of Varnish being novel and well-optimized, it appears that Varnish is pretty conventionally optimized and Squid is just a god-awful piece of development.&lt;p&gt;As he says, this is a 2006 architecture. VM, cache line-size d working set, MRU scheduling -- this is stuff that&apos;s been around for a while. If he were writing this article about a 2010 architecture it would look a bit different. For example, trying to minimize memory access is a laudable goal but in the normal data sets the cache is usually overwhelmed pretty quickly. A 2010 architecture would feature NUMA optimization, although his practice of allocating thread data on the thread stack actually helps this a fair bit (inadvertent?). If Varnish uses threading at a significant level, this kind of optimization is only going to get more significant over time. The modern CPU architecture product cycle goes through a kind of give and take -- it alternates between having just enough fast memory and not enough. Right now, we are just about smack dab in the middle of &quot;just enough,&quot; but the next Intel release is projected to move to &quot;not enough.&quot; Threaded programs in NUMA start to matter a whole lot more on 16, 32, 64 cores.&lt;p&gt;P.S. We actually just ignore 32-bit architectures. Even modern commodity CPUs don&apos;t have them and anyone running a server architecture with only 4 GB of RAM has bigger problems than the proxy software.&lt;p&gt;Edit: Just saw antirez&apos;s post: very good. The VM architecture is good for general case, but it really wasn&apos;t designed for specialty applications. There are many operations that a VM can make if it knows a fair bit about the working set and the memory pattern, but that&apos;s not possible in a generic OS VM (prefetching and streaming are big ones).</text></comment>