<story><title>Deep Learning with Spark and TensorFlow</title><url>https://databricks.com/blog/2016/01/25/deep-learning-with-spark-and-tensorflow.html</url></story><parent_chain><item><author>amelius</author><text>I have a question about neural networks.&lt;p&gt;Say, you are training a NN to recognize handwritten characters 0 and 1, and you have 1000 training images for each character (so 2000 images in total). All images are bitmaps with 0 for black and 1 for white.&lt;p&gt;Now, by accident, all the &amp;quot;0&amp;quot; training-images have an even number of black pixels, and all the &amp;quot;1&amp;quot; training-images have an odd number of black pixels.&lt;p&gt;How do you know that the NN really learns to recognize 0&amp;#x27;s and 1&amp;#x27;s, as opposed to recognizing whether the number of pixels in an image is even or odd?</text></item></parent_chain><comment><author>mindcrime</author><text>I would say that if you&amp;#x27;re using a single layer NN, the answer is &amp;quot;you don&amp;#x27;t really know&amp;quot;.  And that gets to a point about how we still don&amp;#x27;t entirely understand &lt;i&gt;how&lt;/i&gt; neural networks work, even when they do work.&lt;p&gt;If you were using a deep network though, and if the current theory is correct, it would be a slightly different story.  The current thinking, as I understand it, is that with deep networks, each layer learns representations of certain features (say &amp;quot;slashes&amp;quot;, &amp;quot;edges&amp;quot;, &amp;quot;right slanted lines&amp;quot;, &amp;quot;left slanted lines&amp;quot;, etc.) and the progressively higher layers learn representations composed from those more primitive features.   So if a deep net were recognizing your handwritten characters, you could probably reason that it isn&amp;#x27;t &lt;i&gt;just&lt;/i&gt; considering whether the number of black pixels is even or odd.&lt;p&gt;Now in reality this is a pretty contrived, and probably unlikely scenario.  But it&amp;#x27;s a valid question, because there&amp;#x27;s a deeper point to all of this, which involves transference of learning.  That is, how do you take the learning done by a neural network - trained to do one thing - and then leverage that learning in another application.   We still don&amp;#x27;t exactly know how to do that, and that&amp;#x27;s in part because we don&amp;#x27;t entirely understand the nature of the representations the networks build up.   So a very good answer to your question would arguably help understand how to do transference, which would make NN&amp;#x27;s even more useful.</text></comment>
<story><title>Deep Learning with Spark and TensorFlow</title><url>https://databricks.com/blog/2016/01/25/deep-learning-with-spark-and-tensorflow.html</url></story><parent_chain><item><author>amelius</author><text>I have a question about neural networks.&lt;p&gt;Say, you are training a NN to recognize handwritten characters 0 and 1, and you have 1000 training images for each character (so 2000 images in total). All images are bitmaps with 0 for black and 1 for white.&lt;p&gt;Now, by accident, all the &amp;quot;0&amp;quot; training-images have an even number of black pixels, and all the &amp;quot;1&amp;quot; training-images have an odd number of black pixels.&lt;p&gt;How do you know that the NN really learns to recognize 0&amp;#x27;s and 1&amp;#x27;s, as opposed to recognizing whether the number of pixels in an image is even or odd?</text></item></parent_chain><comment><author>JoeAltmaier</author><text>They famously do things like that all the time - instead of recognizing a tank in the woods, they notice all the tanks-in-woods pictures were taken on a sunny day; they train to recognize a sunny day.</text></comment>