<story><title>GPU utilization can be a misleading metric</title><url>https://trainy.ai/blog/gpu-utilization-misleading</url></story><parent_chain></parent_chain><comment><author>SnowflakeOnIce</author><text>&amp;gt; you can get 100% GPU utilization by just reading&amp;#x2F;writing to memory while doing 0 computations&lt;p&gt;Indeed! Utilization is a proxy for what you actually want (which is good use of available hardware). 100% GPU utilization doesn&amp;#x27;t actually indicate this.&lt;p&gt;On the other hand, if you &lt;i&gt;aren&amp;#x27;t&lt;/i&gt; getting 100% GPU utilization, you aren&amp;#x27;t making good use of the hardware.</text></comment>
<story><title>GPU utilization can be a misleading metric</title><url>https://trainy.ai/blog/gpu-utilization-misleading</url></story><parent_chain></parent_chain><comment><author>antognini</author><text>When understanding the performance of your model it&amp;#x27;s very helpful to look at a roofline plot [1].  The roofline plot will show you the floating-point performance as a function of arithmetic intensity for the various ops in your model.  The plot has two regimes: a memory-bound regime on the left and a compute-bound regime on the right.  This can help to identify memory-bound ops that are taking a significant fraction of compute time.&lt;p&gt;[1]: &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Roofline_model&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;en.wikipedia.org&amp;#x2F;wiki&amp;#x2F;Roofline_model&lt;/a&gt;</text></comment>