<story><title>Memory Bandwidth</title><url>https://fgiesen.wordpress.com/2017/04/11/memory-bandwidth/</url></story><parent_chain><item><author>drewg123</author><text>Memory bandwidth and latency is a real problem.  As the referenced article points out, both have gotten worse as compared to CPU speeds as time goes on.&lt;p&gt;The biggest bottleneck in my favorite workload (Netflix CDN server) is memory bandwidth.  When doing software kTLS, we do 4 access per byte served to customers. One access to DMA the contents of a video file to DRAM from storage.  A second access to read the byte for encryption.  A third to write the encrypted data to DRAM.  And a fourth to DMA the encrypted data to the NIC.    That means we&amp;#x27;re bottlenecked serving Netflix video at ~240Gb&amp;#x2F;s on AMD Rome servers (quad channel DDR4-3200, single socket).&lt;p&gt;The theoretical performance, assuming everything besides serving the video is &amp;quot;free&amp;quot; is ~260-300Gb&amp;#x2F;s (depending on numa config, etc).&lt;p&gt;Every cache miss that I can eliminate leads to a small performance increase by both decreasing the memory system load (and leaving more headroom for the workload), &lt;i&gt;and&lt;/i&gt; by eliminating a CPU stall on DRAM access.&lt;p&gt;I&amp;#x27;m thrilled that next generation servers will have DDR5.  That should help a lot with bandwidth, but the PCIe and network speeds will go up as well (Gen4-&amp;gt;Gen5, 100GbE -&amp;gt; 400GbE), so we&amp;#x27;ll still be fighting the same battle.&lt;p&gt;Things like inline-kTLS (where the NIC does the TLS encryption) reduce the memory bandwidth requirements by roughly half, and is how we&amp;#x27;re able to serve up over 350Gb&amp;#x2F;s from a single socket.</text></item></parent_chain><comment><author>magicalhippo</author><text>&amp;gt; Things like inline-kTLS (where the NIC does the TLS encryption) reduce the memory bandwidth requirements by roughly half&lt;p&gt;If storage is accessed via PCI-e, couldn&amp;#x27;t one just pipe it straight from the storage to the NIC, using p2pdma[1][2] or similar? Seems you could skip all of the main DRAM round-trip this way.&lt;p&gt;Or aren&amp;#x27;t things that simple?&lt;p&gt;[1]: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.kernel.org&amp;#x2F;doc&amp;#x2F;html&amp;#x2F;latest&amp;#x2F;driver-api&amp;#x2F;pci&amp;#x2F;p2pdma.html&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.kernel.org&amp;#x2F;doc&amp;#x2F;html&amp;#x2F;latest&amp;#x2F;driver-api&amp;#x2F;pci&amp;#x2F;p2pdma...&lt;/a&gt;&lt;p&gt;[2]: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;youtu.be&amp;#x2F;LDOlqgUZtHE?t=612&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;youtu.be&amp;#x2F;LDOlqgUZtHE?t=612&lt;/a&gt;</text></comment>
<story><title>Memory Bandwidth</title><url>https://fgiesen.wordpress.com/2017/04/11/memory-bandwidth/</url></story><parent_chain><item><author>drewg123</author><text>Memory bandwidth and latency is a real problem.  As the referenced article points out, both have gotten worse as compared to CPU speeds as time goes on.&lt;p&gt;The biggest bottleneck in my favorite workload (Netflix CDN server) is memory bandwidth.  When doing software kTLS, we do 4 access per byte served to customers. One access to DMA the contents of a video file to DRAM from storage.  A second access to read the byte for encryption.  A third to write the encrypted data to DRAM.  And a fourth to DMA the encrypted data to the NIC.    That means we&amp;#x27;re bottlenecked serving Netflix video at ~240Gb&amp;#x2F;s on AMD Rome servers (quad channel DDR4-3200, single socket).&lt;p&gt;The theoretical performance, assuming everything besides serving the video is &amp;quot;free&amp;quot; is ~260-300Gb&amp;#x2F;s (depending on numa config, etc).&lt;p&gt;Every cache miss that I can eliminate leads to a small performance increase by both decreasing the memory system load (and leaving more headroom for the workload), &lt;i&gt;and&lt;/i&gt; by eliminating a CPU stall on DRAM access.&lt;p&gt;I&amp;#x27;m thrilled that next generation servers will have DDR5.  That should help a lot with bandwidth, but the PCIe and network speeds will go up as well (Gen4-&amp;gt;Gen5, 100GbE -&amp;gt; 400GbE), so we&amp;#x27;ll still be fighting the same battle.&lt;p&gt;Things like inline-kTLS (where the NIC does the TLS encryption) reduce the memory bandwidth requirements by roughly half, and is how we&amp;#x27;re able to serve up over 350Gb&amp;#x2F;s from a single socket.</text></item></parent_chain><comment><author>PragmaticPulp</author><text>That&amp;#x27;s a fascinating project to work on.&lt;p&gt;&amp;gt; That means we&amp;#x27;re bottlenecked serving Netflix video at ~240Gb&amp;#x2F;s on AMD Rome servers (quad channel DDR4-3200, single socket).&lt;p&gt;AMD Rome supports octa-channel DDR4 even in single-socket servers. Did you mean octa-channel DDR4, or is there some other limitation in the system that limits you to quad-channel configurations?</text></comment>