<story><title>Run CUDA, unmodified, on AMD GPUs</title><url>https://docs.scale-lang.com/</url></story><parent_chain><item><author>gaogao</author><text>Engineers who are really, really good at CUDA are worth their weight in gold, so there&amp;#x27;s more projects for them than they have time. Worth their weight in gold isn&amp;#x27;t figurative here – the one I know has a ski house more expensive than 180 lbs of gold (~$5,320,814).</text></item><item><author>bootsmann</author><text>Don’t most orgs that are deep enough to run custom cuda kernels have dedicated engineers for this stuff. I can’t imagine a person who can write raw cuda not being able to handle things more difficult than pip install.</text></item><item><author>currymj</author><text>if you&amp;#x27;re talking about building anything, that is already too hard for ML researchers.&lt;p&gt;you have to be able to pip install something and just have it work, reasonably fast, without crashing, and also it has to not interfere with 100 other weird poorly maintained ML library dependencies.</text></item><item><author>eslaught</author><text>Are you aware of HIP? It&amp;#x27;s officially supported and, for code that avoids obscure features of CUDA like inline PTX, it&amp;#x27;s pretty much a find-and-replace to get a working build:&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;ROCm&amp;#x2F;HIP&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;ROCm&amp;#x2F;HIP&lt;/a&gt;&lt;p&gt;Don&amp;#x27;t believe me? Include this at the top of your CUDA code, build with hipcc, and see what happens:&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;gitlab.com&amp;#x2F;StanfordLegion&amp;#x2F;legion&amp;#x2F;-&amp;#x2F;blob&amp;#x2F;master&amp;#x2F;runtime&amp;#x2F;hip_cuda_compat&amp;#x2F;hip_cuda.h&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;gitlab.com&amp;#x2F;StanfordLegion&amp;#x2F;legion&amp;#x2F;-&amp;#x2F;blob&amp;#x2F;master&amp;#x2F;runti...&lt;/a&gt;&lt;p&gt;It&amp;#x27;s incomplete because I&amp;#x27;m lazy but you can see most things are just a single #ifdef away in the implementation.</text></item><item><author>modeless</author><text>A lot of people think AMD should support these translation layers but I think it&amp;#x27;s a bad idea. CUDA is not designed to be vendor agnostic and Nvidia can make things arbitrarily difficult both technically and legally. For example I think it would be against the license agreement of cuDNN or cuBLAS to run them on this. So those and other Nvidia libraries would become part of the API boundary that AMD would need to reimplement and support.&lt;p&gt;Chasing bug-for-bug compatibility is a fool&amp;#x27;s errand. The important users of CUDA are open source. AMD can implement support directly in the upstream projects like pytorch or llama.cpp. And once support is there it can be maintained by the community.</text></item></parent_chain><comment><author>Willish42</author><text>The fact that &amp;quot;worth their weight in cold&amp;quot; typically means in the single-digit millions is &lt;i&gt;fascinating&lt;/i&gt; to me (though I doubt I&amp;#x27;ll be able to get there myself, maybe someday). I looked it up though and I think this is undercounting the current value of gold per ounce&amp;#x2F;lb&amp;#x2F;etc.&lt;p&gt;5320814 &amp;#x2F; 180 &amp;#x2F; 16 = ~1847.5&lt;p&gt;Per &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.apmex.com&amp;#x2F;gold-price&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.apmex.com&amp;#x2F;gold-price&lt;/a&gt; and &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;goldprice.org&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;goldprice.org&amp;#x2F;&lt;/a&gt;, current value is north of $2400 &amp;#x2F; oz. It was around $1800 in 2020. That growth for _gold_ of all things  (up 71% in the last 5 years) is crazy to me.&lt;p&gt;It&amp;#x27;s worth noting that anyone with a ski house that expensive probably has a net worth well over twice the price of that ski house. I guess it&amp;#x27;s time to start learning CUDA!</text></comment>
<story><title>Run CUDA, unmodified, on AMD GPUs</title><url>https://docs.scale-lang.com/</url></story><parent_chain><item><author>gaogao</author><text>Engineers who are really, really good at CUDA are worth their weight in gold, so there&amp;#x27;s more projects for them than they have time. Worth their weight in gold isn&amp;#x27;t figurative here – the one I know has a ski house more expensive than 180 lbs of gold (~$5,320,814).</text></item><item><author>bootsmann</author><text>Don’t most orgs that are deep enough to run custom cuda kernels have dedicated engineers for this stuff. I can’t imagine a person who can write raw cuda not being able to handle things more difficult than pip install.</text></item><item><author>currymj</author><text>if you&amp;#x27;re talking about building anything, that is already too hard for ML researchers.&lt;p&gt;you have to be able to pip install something and just have it work, reasonably fast, without crashing, and also it has to not interfere with 100 other weird poorly maintained ML library dependencies.</text></item><item><author>eslaught</author><text>Are you aware of HIP? It&amp;#x27;s officially supported and, for code that avoids obscure features of CUDA like inline PTX, it&amp;#x27;s pretty much a find-and-replace to get a working build:&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;ROCm&amp;#x2F;HIP&quot;&gt;https:&amp;#x2F;&amp;#x2F;github.com&amp;#x2F;ROCm&amp;#x2F;HIP&lt;/a&gt;&lt;p&gt;Don&amp;#x27;t believe me? Include this at the top of your CUDA code, build with hipcc, and see what happens:&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;gitlab.com&amp;#x2F;StanfordLegion&amp;#x2F;legion&amp;#x2F;-&amp;#x2F;blob&amp;#x2F;master&amp;#x2F;runtime&amp;#x2F;hip_cuda_compat&amp;#x2F;hip_cuda.h&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;gitlab.com&amp;#x2F;StanfordLegion&amp;#x2F;legion&amp;#x2F;-&amp;#x2F;blob&amp;#x2F;master&amp;#x2F;runti...&lt;/a&gt;&lt;p&gt;It&amp;#x27;s incomplete because I&amp;#x27;m lazy but you can see most things are just a single #ifdef away in the implementation.</text></item><item><author>modeless</author><text>A lot of people think AMD should support these translation layers but I think it&amp;#x27;s a bad idea. CUDA is not designed to be vendor agnostic and Nvidia can make things arbitrarily difficult both technically and legally. For example I think it would be against the license agreement of cuDNN or cuBLAS to run them on this. So those and other Nvidia libraries would become part of the API boundary that AMD would need to reimplement and support.&lt;p&gt;Chasing bug-for-bug compatibility is a fool&amp;#x27;s errand. The important users of CUDA are open source. AMD can implement support directly in the upstream projects like pytorch or llama.cpp. And once support is there it can be maintained by the community.</text></item></parent_chain><comment><author>eigenvalue</author><text>That’s pretty funny. Good test of value across the millennia. I wonder if the best aqueduct engineers during the peak of Ancient Rome’s power had villas worth their body weight in gold.</text></comment>