<story><title>Show HN: DeepRhyme (D-Prime) – Generating dope rhymes with machine learning</title><url>https://swarbrickjones.wordpress.com/2016/11/07/deeprhyme-d-prime-generating-dope-rhymes-with-deep-learning/</url></story><parent_chain><item><author>jcoffland</author><text>To the uninitiated these may seem like reasonable results because to the uninitiated hip hop is just a bunch of unintelligible words strung together anyway but these lyrics don&amp;#x27;t make much sense.  There&amp;#x27;s no cohesive stream of thought behind the lines.  It&amp;#x27;s a semirandom juxtaposition of phrases and a great example of one of the biggest limitations of deep learning softwares.  Even the best are only able to maintain very limited context.  This is not just a matter of scaling up to larger networks.  Current methods require exponentially larger networks to achieve linear increases in context handling.  So then why can our brains do it?  The answer is simply that our brains are more than neural networks.</text></item></parent_chain><comment><author>Florin_Andrei</author><text>&amp;gt; &lt;i&gt;So then why can our brains do it?&lt;/i&gt;&lt;p&gt;Because current NNs only simulate, like, less than 1 mm^3 of brain matter. Someone writing lyrics for a song has millions of such tiny networks working concurrently in their brain - and then there are higher-level networks supervising and aggregating the smaller nets, and so on.&lt;p&gt;Current AI NN architectures are flat and have no high-level structure. There&amp;#x27;s no hierarchy. There&amp;#x27;s no plurisemantic context spanning large time intervals and logic trees. No workable memory organized short-, mid- and long-term. Etc, etc, etc.&lt;p&gt;We&amp;#x27;re not even scratching the surface yet.</text></comment>
<story><title>Show HN: DeepRhyme (D-Prime) – Generating dope rhymes with machine learning</title><url>https://swarbrickjones.wordpress.com/2016/11/07/deeprhyme-d-prime-generating-dope-rhymes-with-deep-learning/</url></story><parent_chain><item><author>jcoffland</author><text>To the uninitiated these may seem like reasonable results because to the uninitiated hip hop is just a bunch of unintelligible words strung together anyway but these lyrics don&amp;#x27;t make much sense.  There&amp;#x27;s no cohesive stream of thought behind the lines.  It&amp;#x27;s a semirandom juxtaposition of phrases and a great example of one of the biggest limitations of deep learning softwares.  Even the best are only able to maintain very limited context.  This is not just a matter of scaling up to larger networks.  Current methods require exponentially larger networks to achieve linear increases in context handling.  So then why can our brains do it?  The answer is simply that our brains are more than neural networks.</text></item></parent_chain><comment><author>gavinpc</author><text>I made a similar point in defense of Shakespeare[0], and one reply pointed out a kind of category error on my part:&lt;p&gt;&amp;gt; The samples from all of the examples are nonsense. What&amp;#x27;s interesting is that they, mostly, follow the &lt;i&gt;form&lt;/i&gt; of the original.&lt;p&gt;These examples appear much less nonsensical than those, but that is (I&amp;#x27;d hypothesize) because rap is so much more grammatically and rhetorically liberal.  It does make me wonder how we would distinguish &amp;quot;true&amp;quot; semantics from (trained) formal imitations, when the latter are growing in sophistication.&lt;p&gt;Still, it&amp;#x27;s a danger in this kind of article that, to quote again from that thread,&lt;p&gt;&amp;gt; it&amp;#x27;s a usual tendency of NNs to produce output that looks meaningful to non-experts, yet is complete gibberish to experts.&lt;p&gt;[0] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=12338081&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;news.ycombinator.com&amp;#x2F;item?id=12338081&lt;/a&gt;</text></comment>