<story><title>When not to use deep learning</title><url>http://hyperparameter.space/blog/when-not-to-use-deep-learning/</url></story><parent_chain><item><author>peterburkimsher</author><text>My naïve understanding of deep learning is that it works by finding patterns in the answers, instead of actually solving problems.&lt;p&gt;If I take a multiple-choice exam and always answer &amp;quot;C&amp;quot;, then I have a good chance at getting more than 25%.&lt;p&gt;For image recognition, I think the classifier is doing the real work (trying to actually answer the question), and the deep learning is just seeing if the answer matches the pattern of expected answers.&lt;p&gt;Somehow, this actually works. I think that it&amp;#x27;s because true randomness is hard to find.&lt;p&gt;The problem that I&amp;#x27;ve found is that it&amp;#x27;s really difficult to teach deep learning. I&amp;#x27;m making a Chinese-English teaching tool ( &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;pingtype.github.io&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;pingtype.github.io&lt;/a&gt; ) and sourcing my translations from Google Translate. I find a lot of mistakes in my dictionary that obviously came from Google&amp;#x27;s model getting the word spacing wrong. I can fix it in my own dictionary immediately. If I submit the correction to Google, it just changes some weightings, and hundreds of people will have to submit the same correction before their deep learning will finally catch on that it needs to change something.</text></item></parent_chain><comment><author>daddyo</author><text>Your naive understanding is supported by at least one deep learning authority:&lt;p&gt;&amp;gt; I haven’t found a way to properly articulate this yet but somehow everything we do in deep learning is memorization (interpolation, pattern recognition, etc) instead of thinking (extrapolation, induction, etc). I haven’t seen a single compelling example of a neural network that I would say “thinks”, in a very abstract and hard-to-define feeling of what properties that would have and what that would look like.&lt;p&gt;&amp;gt; All the while I&amp;#x27;m thinking: this thinking process this person goes through as he analyzes this data: THAT is what Machine Learning SHOULD do&lt;p&gt;-- Andrej Karpathy&lt;p&gt;Deep learning for image recognition works because our visual world is made up of structured hierarchical features: Dark&amp;#x2F;Light, Texture, Edge, Part of Object, Object, Scene. Deep learning layers create increasingly higher-level features in a computationally feasible way.</text></comment>
<story><title>When not to use deep learning</title><url>http://hyperparameter.space/blog/when-not-to-use-deep-learning/</url></story><parent_chain><item><author>peterburkimsher</author><text>My naïve understanding of deep learning is that it works by finding patterns in the answers, instead of actually solving problems.&lt;p&gt;If I take a multiple-choice exam and always answer &amp;quot;C&amp;quot;, then I have a good chance at getting more than 25%.&lt;p&gt;For image recognition, I think the classifier is doing the real work (trying to actually answer the question), and the deep learning is just seeing if the answer matches the pattern of expected answers.&lt;p&gt;Somehow, this actually works. I think that it&amp;#x27;s because true randomness is hard to find.&lt;p&gt;The problem that I&amp;#x27;ve found is that it&amp;#x27;s really difficult to teach deep learning. I&amp;#x27;m making a Chinese-English teaching tool ( &lt;a href=&quot;http:&amp;#x2F;&amp;#x2F;pingtype.github.io&quot; rel=&quot;nofollow&quot;&gt;http:&amp;#x2F;&amp;#x2F;pingtype.github.io&lt;/a&gt; ) and sourcing my translations from Google Translate. I find a lot of mistakes in my dictionary that obviously came from Google&amp;#x27;s model getting the word spacing wrong. I can fix it in my own dictionary immediately. If I submit the correction to Google, it just changes some weightings, and hundreds of people will have to submit the same correction before their deep learning will finally catch on that it needs to change something.</text></item></parent_chain><comment><author>shadowmint</author><text>Don&amp;#x27;t just down vote this guys, that doesn&amp;#x27;t help.&lt;p&gt;No, your naive understanding is not correct.&lt;p&gt;&amp;#x27;Deep learning&amp;#x27;, and by that I mean a neural network, works at a super high level by generalising some input (say X) into some related output (lets say, Y).&lt;p&gt;It&amp;#x27;s not just random choice; it&amp;#x27;s like defining a programming function:&lt;p&gt;&lt;pre&gt;&lt;code&gt;    foo(x) -&amp;gt; y { ... }
&lt;/code&gt;&lt;/pre&gt;
Where the &amp;#x27;...&amp;#x27; is implemented by a set of statistical weights and training data and so on.&lt;p&gt;...but ultimately, you raise a valid point.&lt;p&gt;Training models is incredibly time consuming, and incorporating &amp;#x27;corrections&amp;#x27; as new training data is extremely non trivial.&lt;p&gt;It&amp;#x27;s one of the issues with deep learning.&lt;p&gt;I would even go as far as to say, &amp;#x27;You need to regularly update your training data with new examples and counter examples&amp;#x27; as a &amp;#x27;when not to use deep learning&amp;#x27;.</text></comment>