<story><title>Geoffrey Hinton publishes new deep learning algorithm</title><url>https://www.infoq.com/news/2023/01/hinton-forward-algorithm/</url></story><parent_chain></parent_chain><comment><author>jasonjmcghee</author><text>Maybe I&amp;#x27;m missing something, but from the paper &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.cs.toronto.edu&amp;#x2F;~hinton&amp;#x2F;FFA13.pdf&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.cs.toronto.edu&amp;#x2F;~hinton&amp;#x2F;FFA13.pdf&lt;/a&gt;, they use non-conv nets on CIFAR-10 for back prop, resulting in 63% accuracy. And FF achieves 59% accuracy (at best).&lt;p&gt;Those are relatively close figures, but good accuracy on CIFAR-10 is 99%+ and getting ~94% is trivial.&lt;p&gt;So, if an improper architecture for a problem is used and the accuracy is poor, how compelling is using another  optimization approach and achieving similar accuracy?&lt;p&gt;It&amp;#x27;s a unique and interesting approach, but the article specifically mentions it gets accuracy similar to backprop, but if this is the experiment that claim is based on, it loses some credibility in my eyes.</text></comment>
<story><title>Geoffrey Hinton publishes new deep learning algorithm</title><url>https://www.infoq.com/news/2023/01/hinton-forward-algorithm/</url></story><parent_chain></parent_chain><comment><author>goethes_kind</author><text>I skimmed through the paper and am a bit confused. There&amp;#x27;s only one equation and I feel like he rushed to publish a shower thought without even bothering to flesh it out mathematically.&lt;p&gt;So how do you optimize a layer? Do you still use gradient descent? So you are have a per layer loss with a positive and negative component and then do gradient descent?&lt;p&gt;So then what is the label for each layer? Do you use the same label for each layer?&lt;p&gt;And what does he mean by the forward pass not being fully known? I don&amp;#x27;t get this application of the blackbox between layers. Why would you want to do that?</text></comment>