<story><title>An Introduction to Recurrent Neural Networks</title><url>https://victorzhou.com/blog/intro-to-rnns/</url></story><parent_chain></parent_chain><comment><author>stereolambda</author><text>It&amp;#x27;s worth noting that apparently (as I learned lately) RNNs are going slightly out of fashion because they are hard to parallelize and have trouble remembering important stuff at larger distances. Transformers are proposed as a possible solution - very roughly speaking, they use attention mechanisms instead of recurrent memory and can run in parallel.&lt;p&gt;I have to say that while I understand the problems with recurrent nets (which I&amp;#x27;ve used many times), I haven&amp;#x27;t yet grokked the alternatives. Here are some decently looking search results for you as starting points. Warning, these can be longer and heavier reads probably not for beginners.&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;towardsdatascience.com&amp;#x2F;the-fall-of-rnn-lstm-2d1594c74ce0&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;towardsdatascience.com&amp;#x2F;the-fall-of-rnn-lstm-2d1594c7...&lt;/a&gt; (there&amp;#x27;s some sensationalism here to be fair)&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;mchromiak.github.io&amp;#x2F;articles&amp;#x2F;2017&amp;#x2F;Sep&amp;#x2F;12&amp;#x2F;Transformer-Attention-is-all-you-need&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;mchromiak.github.io&amp;#x2F;articles&amp;#x2F;2017&amp;#x2F;Sep&amp;#x2F;12&amp;#x2F;Transformer...&lt;/a&gt;&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.analyticsvidhya.com&amp;#x2F;blog&amp;#x2F;2019&amp;#x2F;06&amp;#x2F;understanding-transformers-nlp-state-of-the-art-models&amp;#x2F;&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.analyticsvidhya.com&amp;#x2F;blog&amp;#x2F;2019&amp;#x2F;06&amp;#x2F;understanding-t...&lt;/a&gt;&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;www.tensorflow.org&amp;#x2F;beta&amp;#x2F;tutorials&amp;#x2F;text&amp;#x2F;transformer&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;www.tensorflow.org&amp;#x2F;beta&amp;#x2F;tutorials&amp;#x2F;text&amp;#x2F;transformer&lt;/a&gt;&lt;p&gt;That being said, I think that understanding RNNs is very beneficial conceptually and nowadays there are relatively easy to use implementations that should be pretty good for many use cases.</text></comment>
<story><title>An Introduction to Recurrent Neural Networks</title><url>https://victorzhou.com/blog/intro-to-rnns/</url></story><parent_chain></parent_chain><comment><author>wish5031</author><text>Nice! I like that the author wrote the code by hand rather than leaning on some framework. It makes it a lot easier to connect the math to the code. :)&lt;p&gt;As a meta-comment on these &amp;quot;Introduction to _____ neural network&amp;quot; articles (not just this one), I wish people would spend more time talking about when their neural net isn&amp;#x27;t the right tool for the job. SVMs, kNN, even basic regression techniques aren&amp;#x27;t any less effective than they were 20 years ago. They&amp;#x27;re easier to interpret and debug, require many fewer parameters, and potentially (you may need to apply some tricks here or there) faster at both training and evaluation time.</text></comment>