<story><title>Scientists See Promise in Deep-Learning Programs</title><url>http://nytimes.com/2012/11/24/science/scientists-see-advances-in-deep-learning-a-part-of-artificial-intelligence.html</url></story><parent_chain></parent_chain><comment><author>bravura</author><text>Since I see some misunderstanding about deep learning, let me explain the fundamental idea: It&apos;s about reusing intermediate work.&lt;p&gt;The intuition is let&apos;s say I told you to write a complicated computer program. Let&apos;s say I told you that you could use routines and subroutines, but you couldn&apos;t use subsubroutines, or deeper levels of abstraction. In this restricted case, you &lt;i&gt;could&lt;/i&gt; write any computer program, but you would have to use a lot of code-copying. With arbitrary levels of abstraction, you could do code reuse much more elegantly, and your code would be more compact.&lt;p&gt;Here is a more formal description: If you have a complicated non-linear function, you can describe it similarly to a circuit. If you restrict the depth of the circuit, you can in principle represent any function, but you need a really wide (exponentially wide) circuit. This can lead to overfitting. (Occam&apos;s Razor) By comparison, with a deep circuit, you can represent arbitrary functions &lt;i&gt;compactly&lt;/i&gt;.&lt;p&gt;Standard SVMs and random forests can be shown, mathematically, to have a limited number of layers (circuit depth).&lt;p&gt;It turns out that expressing deep models using neural networks is quite convenient.&lt;p&gt;I gave an introduction to deep learning in 2009 that describes these intuitions: &lt;a href=&quot;http://vimeo.com/7977427&quot; rel=&quot;nofollow&quot;&gt;http://vimeo.com/7977427&lt;/a&gt;</text></comment>
<story><title>Scientists See Promise in Deep-Learning Programs</title><url>http://nytimes.com/2012/11/24/science/scientists-see-advances-in-deep-learning-a-part-of-artificial-intelligence.html</url></story><parent_chain></parent_chain><comment><author>theschwa</author><text>Geoffrey Hinton, mentioned in the article, has his class on neural networks available on Coursera &lt;a href=&quot;https://www.coursera.org/course/neuralnets&quot; rel=&quot;nofollow&quot;&gt;https://www.coursera.org/course/neuralnets&lt;/a&gt;</text></comment>