<story><title>Cache invalidation really is one of the hardest problems in computer science</title><url>https://surfingcomplexity.blog/2022/11/25/cache-invalidation-really-is-one-of-the-hardest-things-in-computer-science/</url></story><parent_chain></parent_chain><comment><author>eric4smith</author><text>Slightly off topic...&lt;p&gt;Back before my Phoenix&amp;#x2F;Elixir days, we used to have to cache almost everything. Now we don&amp;#x27;t cache anything -- even with more traffic and less servers. Now, the projects I work on are not getting millions of views every day. But they are still important for many customers.&lt;p&gt;We don&amp;#x27;t really design anything that needs to be cached anymore. Just better access patterns and table design as we get wiser.&lt;p&gt;Take Wordpress as an example. There really should not be a need to cache a Wordpress site. But some of the most popular plugins are those that cache. To be fair, I suppose, their legacy non-optimal database design access pattern and code is responsible for this.&lt;p&gt;Just getting something up and aiming for product-market fit is priority #1 and then struggling with caching if it gets popular, at which point, more resources can be thrown at it.</text></comment>
<story><title>Cache invalidation really is one of the hardest problems in computer science</title><url>https://surfingcomplexity.blog/2022/11/25/cache-invalidation-really-is-one-of-the-hardest-things-in-computer-science/</url></story><parent_chain></parent_chain><comment><author>dragontamer</author><text>&amp;gt; Cache coherency ensures that the behavior is correct, but every time a cache is invalidated and the same memory has to be retrieved from main memory again, it pays the performance penalty of reading from main memory.&lt;p&gt;1. First no. If any such rereads occur, they will be from the LLC (last-level cache, or L3 cache for Intel&amp;#x2F;AMD CPUs).&lt;p&gt;2. Second no. IIRC, modern caches are snooping and&amp;#x2F;or directory caches. This means that when Core#0 says &amp;quot;I&amp;#x27;m changing cacheline X&amp;quot;, Core#0 knows that Core#1 has it in its L2 (or deeper) caches. So Core#0 will publish the results of that change to Core#1&lt;p&gt;3. The examples they gave are missing read&amp;#x2F;write barriers, which are rather important. #2 will only happen during read&amp;#x2F;write barriers. That is to say: your code is memory-unsafe by default. If order and cache-invalidation matters, you actually need a barrier instruction (!!!) to make sure all these messages are passed to the right place at the right time.&lt;p&gt;---------&lt;p&gt;For better or for worse, modern CPU design is about doing things thread-unsafe by default. If the programmer recognizes a problem may occur, it is the responsibility of the programmer to put the memory barriers in the right place.&lt;p&gt;This is because the CPU is not the only thing that can reorder things... but also the cache system... as well as the compiler. So memory barriers inform the compiler + CPU + cache system when to synchronize.&lt;p&gt;The original Netflix article is more precisely worded.&lt;p&gt;&amp;gt; This consistency is ensured with so-called “cache coherency protocol.”&lt;p&gt;With a link to MESIF (F for Forwarding state, which is one such protocol &amp;#x2F; way to share L1 cache info without going all the way to LLC or DRAM)&lt;p&gt;I&amp;#x27;m pretty sure that GPUs actually invalidate the cache dumbly, without any MESI &amp;#x2F; MESIF (or whatever). GPUs are actually really bad at these kinds of ping-pong operations and synchronization... preferring thread-fences and other synchronization mechanisms instead.&lt;p&gt;------&lt;p&gt;That being said, I think the blogpost is a good introduction to the subject. But note that its a bit imprecise with some of the low level details. I guess its correct for the GPU-world though, so its not completely inaccurate...&lt;p&gt;-----&lt;p&gt;The original Netflix article has an even more difficult problem revolving around Java&amp;#x27;s Superclass cache and how it is affecting &amp;quot;true sharing&amp;quot; of caches. I&amp;#x27;m not very familiar with JVM internals, so I lost track of the discussion at that point.</text></comment>