<story><title>Beyond self-attention: How a small language model predicts the next token</title><url>https://shyam.blog/posts/beyond-self-attention/</url></story><parent_chain></parent_chain><comment><author>tysam_and</author><text>Some of the topics in the parent post should not be a major surprise to anyone who has read &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;people.math.harvard.edu&amp;#x2F;~ctm&amp;#x2F;home&amp;#x2F;text&amp;#x2F;others&amp;#x2F;shannon&amp;#x2F;entropy&amp;#x2F;entropy.pdf&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;people.math.harvard.edu&amp;#x2F;~ctm&amp;#x2F;home&amp;#x2F;text&amp;#x2F;others&amp;#x2F;shanno...&lt;/a&gt; ! If we do not have read the foundations of the field that we are in, we are doomed to be mystified by unexplained phenomena which arise pretty naturally as consequences of already-distilled work!&lt;p&gt;That said, the experiments seem very thorough, on a first pass&amp;#x2F;initial cursory examination, I appreciate the amount of detail that seemed to go into them.&lt;p&gt;The tradeoff between learning existing theory, and attempting to re-derive it from scratch, I think, is a hard tradeoff, as not having the traditional foundation allows for the discovery of new things, but having it allows for a deeper understanding of certain phenomena. There is a tradeoff either way.&lt;p&gt;I&amp;#x27;ve seen several people here in the comments seemingly shocked that a model that maximizes the log likelihood of a sequence given the data somehow does not magically deviate from that behavior when run in inference. It&amp;#x27;s a density estimation model, do you want it to magically recite Shakespeare from the void?&lt;p&gt;Please! Let&amp;#x27;s stick to the basics, it will help experiments like this make much more sense as there already is a very clear mathematical foundation which clearly explains it (and said emergent phenomena).&lt;p&gt;If you want more specifics, there are several layers, Shannon&amp;#x27;s treatment of ergodic systems is a good start (though there is some minor deviation from that here, but it likely is a &amp;#x27;close enough&amp;#x27; match to what&amp;#x27;s happening here to be properly instructive to the reader about the general dynamics of what is going on, overall.)</text></comment>
<story><title>Beyond self-attention: How a small language model predicts the next token</title><url>https://shyam.blog/posts/beyond-self-attention/</url></story><parent_chain></parent_chain><comment><author>kmeisthax</author><text>I had the exact same idea after seeing Google point out that you can[0] get ChatGPT to regurgitate verbatim training data by asking it to repeat the same word over and over again[1]. I&amp;#x27;m glad to see someone else actually bring it to fruition.&lt;p&gt;This, of course, brings two additional questions:&lt;p&gt;1. Is this &amp;quot;AI, hold the AI&amp;quot; approach more energy-efficient than having gradient descent backpropagation compress a bunch of training data into a model that can then be run on specialized AI coprocessors?&lt;p&gt;2. Will this result wind up being evidence in the ongoing lawsuits against OpenAI and Stability AI?&lt;p&gt;[0] Could. OpenAI now blocks generation if you fill the context window with a single word.&lt;p&gt;[1] &lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;arxiv.org&amp;#x2F;abs&amp;#x2F;2311.17035&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;arxiv.org&amp;#x2F;abs&amp;#x2F;2311.17035&lt;/a&gt;</text></comment>