<story><title>Rust and C++ on Floating-Point Intensive Code</title><url>https://www.reidatcheson.com/hpc/architecture/performance/rust/c++/2019/10/19/measure-cache.html</url></story><parent_chain><item><author>diamondlovesyou</author><text>I have some experience with this, ie ensuring LLVM optimizes and codegens the &amp;quot;best&amp;quot;! I have been working to generate &lt;i&gt;target independent&lt;/i&gt; &amp;quot;kernels&amp;quot; for the Rav1e AV1 encoder and have had to do a lot of unidiomatic things to get LLVM to generate machine code similar in quality to hand written ASM. Granted, this is on integers and not floats, but the same principles should apply.&lt;p&gt;What I&amp;#x27;ve found is that you need to ignore most of Rust: use&amp;#x2F;load raw pointers, don&amp;#x27;t use slices, unroll manually, vectorize manually, and check preconditions manually. You&amp;#x27;ll still get the amazing type system, but the code will have to be more C-like than Rust-like.&lt;p&gt;* raw pointers: LLVM is pretty good at optimizing C code. Rust specific optimization needs some work. (edit: I assumed arrays here, so you&amp;#x27;ll need the pointer for offsets; references are still okay. You&amp;#x27;d also use the pointers for iterating instead of the usual slice iteration patterns)&lt;p&gt;* no slices: index checking is expensive, not to the CPU, the CPU rarely misses the check branches, but to the optimizer. I&amp;#x27;ve found these are mostly left un-elided, even after inlining.&lt;p&gt;* no slices: slice indexing uses overflow checking. For Rav1e&amp;#x27;s case, the block&amp;#x2F;plane sizes mean that doing the index calculation using `u32` will never overflow, so calculating the offsets using u32 is fine (I&amp;#x27;ll have to switch to using a pseudo u24 integer for GPUs though, because u32 is still expensive on them).&lt;p&gt;* unroll manually: LLVM would probably do more of this with profiling info, but I&amp;#x27;ve never found it (this is subjective!) to do any unrolling w&amp;#x2F;o. Maybe if all the other items here are also done...&lt;p&gt;* vectorize manually: Similar to unrolling. I&amp;#x27;ve observed only limited automatic vectorization.&lt;p&gt;* And to get safety back: check, check, and check before calling the fast kernel! Ie wrap the kernel function with one that does all the checks elided in the kernel.&lt;p&gt;Source: Wrote &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;xiph&amp;#x2F;rav1e&amp;#x2F;pull&amp;#x2F;1716&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;xiph&amp;#x2F;rav1e&amp;#x2F;pull&amp;#x2F;1716&lt;/a&gt;, which speeds up the non-asm encodes by over 2x!</text></item></parent_chain><comment><author>gameswithgo</author><text>the first order reason Rust and C++ differ in the article is because Rust will not pass the ffastmath flag to llvm, not because of any of this stuff.</text></comment>
<story><title>Rust and C++ on Floating-Point Intensive Code</title><url>https://www.reidatcheson.com/hpc/architecture/performance/rust/c++/2019/10/19/measure-cache.html</url></story><parent_chain><item><author>diamondlovesyou</author><text>I have some experience with this, ie ensuring LLVM optimizes and codegens the &amp;quot;best&amp;quot;! I have been working to generate &lt;i&gt;target independent&lt;/i&gt; &amp;quot;kernels&amp;quot; for the Rav1e AV1 encoder and have had to do a lot of unidiomatic things to get LLVM to generate machine code similar in quality to hand written ASM. Granted, this is on integers and not floats, but the same principles should apply.&lt;p&gt;What I&amp;#x27;ve found is that you need to ignore most of Rust: use&amp;#x2F;load raw pointers, don&amp;#x27;t use slices, unroll manually, vectorize manually, and check preconditions manually. You&amp;#x27;ll still get the amazing type system, but the code will have to be more C-like than Rust-like.&lt;p&gt;* raw pointers: LLVM is pretty good at optimizing C code. Rust specific optimization needs some work. (edit: I assumed arrays here, so you&amp;#x27;ll need the pointer for offsets; references are still okay. You&amp;#x27;d also use the pointers for iterating instead of the usual slice iteration patterns)&lt;p&gt;* no slices: index checking is expensive, not to the CPU, the CPU rarely misses the check branches, but to the optimizer. I&amp;#x27;ve found these are mostly left un-elided, even after inlining.&lt;p&gt;* no slices: slice indexing uses overflow checking. For Rav1e&amp;#x27;s case, the block&amp;#x2F;plane sizes mean that doing the index calculation using `u32` will never overflow, so calculating the offsets using u32 is fine (I&amp;#x27;ll have to switch to using a pseudo u24 integer for GPUs though, because u32 is still expensive on them).&lt;p&gt;* unroll manually: LLVM would probably do more of this with profiling info, but I&amp;#x27;ve never found it (this is subjective!) to do any unrolling w&amp;#x2F;o. Maybe if all the other items here are also done...&lt;p&gt;* vectorize manually: Similar to unrolling. I&amp;#x27;ve observed only limited automatic vectorization.&lt;p&gt;* And to get safety back: check, check, and check before calling the fast kernel! Ie wrap the kernel function with one that does all the checks elided in the kernel.&lt;p&gt;Source: Wrote &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;xiph&amp;#x2F;rav1e&amp;#x2F;pull&amp;#x2F;1716&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;xiph&amp;#x2F;rav1e&amp;#x2F;pull&amp;#x2F;1716&lt;/a&gt;, which speeds up the non-asm encodes by over 2x!</text></item></parent_chain><comment><author>nine_k</author><text>To sum up: LLVM is heavily optimized for C code, not so much for Rust. So Rust code has to imitate certain C mannerisms for the optimizer to kick in.&lt;p&gt;You have to pay the price of compatibility with the existing toolchain even if it&amp;#x27;s not your explicit  goal.</text></comment>