<story><title>GradIEEEnt half decent</title><url>http://tom7.org/grad/</url></story><parent_chain></parent_chain><comment><author>unlikelymordant</author><text>Reminds of some earlier openai work &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;openai.com&amp;#x2F;research&amp;#x2F;nonlinear-computation-in-deep-linear-networks&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;openai.com&amp;#x2F;research&amp;#x2F;nonlinear-computation-in-deep-li...&lt;/a&gt;&lt;p&gt;&amp;quot;Neural networks consist of stacks of a linear layer followed by a nonlinearity like tanh or rectified linear unit. Without the nonlinearity, consecutive linear layers would be in theory mathematically equivalent to a single linear layer. So it’s a surprise that floating point arithmetic is nonlinear enough to yield trainable deep networks.&amp;quot;</text></comment>
<story><title>GradIEEEnt half decent</title><url>http://tom7.org/grad/</url></story><parent_chain></parent_chain><comment><author>chrisldgk</author><text>Always delighted to see tom7 content. 
Dude just thinks out of the box like no one else.</text></comment>