<story><title>Mammals dream about the world they are about to experience before they are born</title><url>https://news.yale.edu/2021/07/22/eyes-wide-shut-how-newborn-mammals-dream-world-theyre-entering</url></story><parent_chain><item><author>chrisseaton</author><text>&amp;gt; Most human babies take 9-18 months to walk, and this is commonly attributed to the fact that human babies have immature brains&lt;p&gt;Have you ever seen a new-born baby&amp;#x27;s legs?&lt;p&gt;They aren&amp;#x27;t even remotely strong enough to walk, no matter what the brain tells them. It takes those months to develop leg muscles. New-borns don&amp;#x27;t have them.&lt;p&gt;It&amp;#x27;s absolutely not restricted by the brain size.</text></item><item><author>teruakohatu</author><text>&amp;gt; Are these innate patterns of activity priming mammalian brains in a similar way?&lt;p&gt;A newborn foal can stand within 55 minutes of being born and can walk or run within 90 minutes. That is crazy fast training speed. Is it training initialization or is it a form of transfer learning?&lt;p&gt;Most human babies take 9-18 months to walk, and this is commonly attributed to the fact that human babies have immature brains that cannot mature due to birth canal constraints, which is constrained by pelvis size which is constrained by the need to walk upright.&lt;p&gt;I have wondered about how true this is. Maybe our brains are just so much more powerful, or capable of much deeper understanding, that we require a much lower initial learning rate.&lt;p&gt;The lottery ticket could explain why some children learn to walk at 6 months, while others are closer to 24 months, with no different in intelligence or motor skills in later life.&lt;p&gt;Edit: As some people below have correctly pointed out, human babies could not walk within an hour for physiological reasons, but nor do they exhibit the basic motor skills, spatial reasoning or image processing required for walking. Many human babies even struggle to feed for the first 24-48 hours or longer.&lt;p&gt;Meanwhile baboon babies can hold onto their mother from birth while the mother climbs trees.</text></item><item><author>gojomo</author><text>Something about this reminds me of the &amp;#x27;Winning Ticket Hypothesis&amp;#x27; in artifical neural networks: that some &amp;#x27;random&amp;#x27; initializations prime a network far better for later faster learning. From &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;arxiv.org&amp;#x2F;abs&amp;#x2F;1803.03635&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;arxiv.org&amp;#x2F;abs&amp;#x2F;1803.03635&lt;/a&gt; - the abstract:&lt;p&gt;&lt;i&gt;Neural network pruning techniques can reduce the parameter counts of trained networks by over 90%, decreasing storage requirements and improving computational performance of inference without compromising accuracy. However, contemporary experience is that the sparse architectures produced by pruning are difficult to train from the start, which would similarly improve training performance.&lt;/i&gt;&lt;p&gt;&lt;i&gt;We find that a standard pruning technique naturally uncovers subnetworks whose initializations made them capable of training effectively. Based on these results, we articulate the &amp;quot;lottery ticket hypothesis:&amp;quot; dense, randomly-initialized, feed-forward networks contain subnetworks (&amp;quot;winning tickets&amp;quot;) that - when trained in isolation - reach test accuracy comparable to the original network in a similar number of iterations. The winning tickets we find have won the initialization lottery: their connections have initial weights that make training particularly effective.&lt;/i&gt;&lt;p&gt;&lt;i&gt;We present an algorithm to identify winning tickets and a series of experiments that support the lottery ticket hypothesis and the importance of these fortuitous initializations. We consistently find winning tickets that are less than 10-20% of the size of several fully-connected and convolutional feed-forward architectures for MNIST and CIFAR10. Above this size, the winning tickets that we find learn faster than the original network and reach higher test accuracy.&lt;/i&gt;&lt;p&gt;Are these innate patterns of activity priming mammalian brains in a similar way?</text></item></parent_chain><comment><author>narag</author><text>&lt;i&gt;It&amp;#x27;s absolutely not restricted by the brain size.&lt;/i&gt;&lt;p&gt;The brain size needs a huge head, so in a sense it is.&lt;p&gt;All the factors are interwined. The growth of the brain seems to have been a very fast evolution, The pelvis had no time to adapt, so giving birth early was the final solution.&lt;p&gt;With that in mind, I don&amp;#x27;t find &lt;i&gt;fair&lt;/i&gt; to shame the pelvis, or the brain. The weakest link is the legs, but it&amp;#x27;s not their &lt;i&gt;fault&lt;/i&gt; either since they had no time to mature.&lt;p&gt;Give one of those cocky foals a head of proportional weight and allow them to use only two limbs and see who&amp;#x27;s stumbling now!&lt;p&gt;Edit: newborns can&amp;#x27;t even balance their heads on their necks, they&amp;#x27;re extremely fragile.</text></comment>
<story><title>Mammals dream about the world they are about to experience before they are born</title><url>https://news.yale.edu/2021/07/22/eyes-wide-shut-how-newborn-mammals-dream-world-theyre-entering</url></story><parent_chain><item><author>chrisseaton</author><text>&amp;gt; Most human babies take 9-18 months to walk, and this is commonly attributed to the fact that human babies have immature brains&lt;p&gt;Have you ever seen a new-born baby&amp;#x27;s legs?&lt;p&gt;They aren&amp;#x27;t even remotely strong enough to walk, no matter what the brain tells them. It takes those months to develop leg muscles. New-borns don&amp;#x27;t have them.&lt;p&gt;It&amp;#x27;s absolutely not restricted by the brain size.</text></item><item><author>teruakohatu</author><text>&amp;gt; Are these innate patterns of activity priming mammalian brains in a similar way?&lt;p&gt;A newborn foal can stand within 55 minutes of being born and can walk or run within 90 minutes. That is crazy fast training speed. Is it training initialization or is it a form of transfer learning?&lt;p&gt;Most human babies take 9-18 months to walk, and this is commonly attributed to the fact that human babies have immature brains that cannot mature due to birth canal constraints, which is constrained by pelvis size which is constrained by the need to walk upright.&lt;p&gt;I have wondered about how true this is. Maybe our brains are just so much more powerful, or capable of much deeper understanding, that we require a much lower initial learning rate.&lt;p&gt;The lottery ticket could explain why some children learn to walk at 6 months, while others are closer to 24 months, with no different in intelligence or motor skills in later life.&lt;p&gt;Edit: As some people below have correctly pointed out, human babies could not walk within an hour for physiological reasons, but nor do they exhibit the basic motor skills, spatial reasoning or image processing required for walking. Many human babies even struggle to feed for the first 24-48 hours or longer.&lt;p&gt;Meanwhile baboon babies can hold onto their mother from birth while the mother climbs trees.</text></item><item><author>gojomo</author><text>Something about this reminds me of the &amp;#x27;Winning Ticket Hypothesis&amp;#x27; in artifical neural networks: that some &amp;#x27;random&amp;#x27; initializations prime a network far better for later faster learning. From &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;arxiv.org&amp;#x2F;abs&amp;#x2F;1803.03635&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;arxiv.org&amp;#x2F;abs&amp;#x2F;1803.03635&lt;/a&gt; - the abstract:&lt;p&gt;&lt;i&gt;Neural network pruning techniques can reduce the parameter counts of trained networks by over 90%, decreasing storage requirements and improving computational performance of inference without compromising accuracy. However, contemporary experience is that the sparse architectures produced by pruning are difficult to train from the start, which would similarly improve training performance.&lt;/i&gt;&lt;p&gt;&lt;i&gt;We find that a standard pruning technique naturally uncovers subnetworks whose initializations made them capable of training effectively. Based on these results, we articulate the &amp;quot;lottery ticket hypothesis:&amp;quot; dense, randomly-initialized, feed-forward networks contain subnetworks (&amp;quot;winning tickets&amp;quot;) that - when trained in isolation - reach test accuracy comparable to the original network in a similar number of iterations. The winning tickets we find have won the initialization lottery: their connections have initial weights that make training particularly effective.&lt;/i&gt;&lt;p&gt;&lt;i&gt;We present an algorithm to identify winning tickets and a series of experiments that support the lottery ticket hypothesis and the importance of these fortuitous initializations. We consistently find winning tickets that are less than 10-20% of the size of several fully-connected and convolutional feed-forward architectures for MNIST and CIFAR10. Above this size, the winning tickets that we find learn faster than the original network and reach higher test accuracy.&lt;/i&gt;&lt;p&gt;Are these innate patterns of activity priming mammalian brains in a similar way?</text></item></parent_chain><comment><author>outworlder</author><text>True. But the movements on all limbs are quite uncoordinated at first. It&amp;#x27;s also not restricted to muscle strength.</text></comment>