<story><title>Too much efficiency makes everything worse (2022)</title><url>https://sohl-dickstein.github.io/2022/11/06/strong-Goodhart.html</url></story><parent_chain><item><author>refibrillator</author><text>I recognize the author Jascha as an incredibly brilliant ML researcher, formerly at Google  Brain and now at Anthropic.&lt;p&gt;Among his notable accomplishments, he and coauthors mathematically characterized the propagation of signals through deep neural networks via techniques from physics and statistics (mean field and free probability theory). Leading to arguably some of the most profound yet under-appreciated theoretical and experimental results in ML in the past decade. For example see “dynamical isometry” [1] and the evolution of those ideas which were instrumental in achieving convergence in very deep transformer models [2].&lt;p&gt;After reading this post and the examples given, in my eyes there is no question that this guy has an extraordinary intuition for optimization, spanning beyond the boundaries of ML and across the fabric of modern society.&lt;p&gt;We ought to recognize his technical background and raise this discussion above quibbles about semantics and definitions.&lt;p&gt;Let’s address the heart of his message, the very human and empathetic call to action that stands in the shadow of rapid technological progress:&lt;p&gt;&lt;i&gt;&amp;gt; If you are a scientist looking for research ideas which are pro-social, and have the potential to create a whole new field, you should consider building formal (mathematical) bridges between results on overfitting in machine learning, and problems in economics, political science, management science, operations research, and elsewhere.&lt;/i&gt;&lt;p&gt;[1] Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks&lt;p&gt;&lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;proceedings.mlr.press&amp;#x2F;v80&amp;#x2F;xiao18a&amp;#x2F;xiao18a.pdf&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;proceedings.mlr.press&amp;#x2F;v80&amp;#x2F;xiao18a&amp;#x2F;xiao18a.pdf&lt;/a&gt;&lt;p&gt;[2] ReZero is All You Need: Fast Convergence at Large Depth&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;arxiv.org&amp;#x2F;pdf&amp;#x2F;2003.04887&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;arxiv.org&amp;#x2F;pdf&amp;#x2F;2003.04887&lt;/a&gt;</text></item></parent_chain><comment><author>tablatom</author><text>Interesting timing for me! Just a couple of days ago I discovered the work of biologist Olivier Hamant who has been raising exactly this issue. His main thesis is that very high performance (which he defines as efficacy towards a known goal plus efficiency) and very high robustness (the ability to withstand large fluctuations in the system) are physically incompatible. Examples abound in nature. Contrary to common perception evolution does not optimise for high performance but high robustness. Giving priority to performance may have made sense in a world of abundant resources, but we are now facing a very different period where instability is the norm. We must (and will be forced to) backtrack on performance in order to become robust. It’s the freshest and most interesting take on the poly-crisis that I’ve seen in a long time.&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;books.google.co.uk&amp;#x2F;books&amp;#x2F;about&amp;#x2F;Tracts_N_50_Antidote_to_the_cult_of_perf.html?id=XvQMEQAAQBAJ&amp;amp;redir_esc=y&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;books.google.co.uk&amp;#x2F;books&amp;#x2F;about&amp;#x2F;Tracts_N_50_Antidote_...&lt;/a&gt;</text></comment>
<story><title>Too much efficiency makes everything worse (2022)</title><url>https://sohl-dickstein.github.io/2022/11/06/strong-Goodhart.html</url></story><parent_chain><item><author>refibrillator</author><text>I recognize the author Jascha as an incredibly brilliant ML researcher, formerly at Google  Brain and now at Anthropic.&lt;p&gt;Among his notable accomplishments, he and coauthors mathematically characterized the propagation of signals through deep neural networks via techniques from physics and statistics (mean field and free probability theory). Leading to arguably some of the most profound yet under-appreciated theoretical and experimental results in ML in the past decade. For example see “dynamical isometry” [1] and the evolution of those ideas which were instrumental in achieving convergence in very deep transformer models [2].&lt;p&gt;After reading this post and the examples given, in my eyes there is no question that this guy has an extraordinary intuition for optimization, spanning beyond the boundaries of ML and across the fabric of modern society.&lt;p&gt;We ought to recognize his technical background and raise this discussion above quibbles about semantics and definitions.&lt;p&gt;Let’s address the heart of his message, the very human and empathetic call to action that stands in the shadow of rapid technological progress:&lt;p&gt;&lt;i&gt;&amp;gt; If you are a scientist looking for research ideas which are pro-social, and have the potential to create a whole new field, you should consider building formal (mathematical) bridges between results on overfitting in machine learning, and problems in economics, political science, management science, operations research, and elsewhere.&lt;/i&gt;&lt;p&gt;[1] Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks&lt;p&gt;&lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;proceedings.mlr.press&amp;#x2F;v80&amp;#x2F;xiao18a&amp;#x2F;xiao18a.pdf&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;proceedings.mlr.press&amp;#x2F;v80&amp;#x2F;xiao18a&amp;#x2F;xiao18a.pdf&lt;/a&gt;&lt;p&gt;[2] ReZero is All You Need: Fast Convergence at Large Depth&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;arxiv.org&amp;#x2F;pdf&amp;#x2F;2003.04887&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;arxiv.org&amp;#x2F;pdf&amp;#x2F;2003.04887&lt;/a&gt;</text></item></parent_chain><comment><author>thomasahle</author><text>I love the idea of ReZero, basically using a trainable parameter, alpha, in residual layers like this:&lt;p&gt;&lt;pre&gt;&lt;code&gt;  Deep Network                  | xi+1 = F(xi)                 
  Residual Network              | xi+1 = xi + F(xi)            
  Deep Network + Norm           | xi+1 = Norm(F(xi))           
  Residual Network + Pre-Norm   | xi+1 = xi + F(Norm(xi))      
  Residual Network + Post-Norm  | xi+1 = Norm(xi + F(xi))      
  ReZero                        | xi+1 = xi + αi F(xi)         
&lt;/code&gt;&lt;/pre&gt;
However, I haven&amp;#x27;t actually seen this used in practice.
The papers we have on Gemma and Llama all still seem to be using layer norms.&lt;p&gt;Am I missing something?</text></comment>