<story><title>Better and Faster Large Language Models via Multi-Token Prediction</title><url>https://arxiv.org/abs/2404.19737</url></story><parent_chain><item><author>deskamess</author><text>Side track: There is so much going on this space. I wish there was a chronological flow of a machine learning scenario&amp;#x2F;story with all the terms being introduced as we meet them (data, pre-training, training, inference, mixture of experts, RAG). Like someone walking me through a factory explaining what happens at each stage (like Mr Rogers used to do). Most of the time I do not know where the terms fit in the big picture. When I first came across pre-training I thought it was something done to the data before training happened but it was actually another training.</text></item></parent_chain><comment><author>berkes</author><text>&amp;gt; Most of the time I do not know where the terms fit in the big picture.&lt;p&gt;Nor do the majority of &amp;quot;AI&amp;quot; experts and consultants that I see on LinkedIn, Twitter or in podcasts.&lt;p&gt;The S&amp;#x2F;N ratio is very low in this field. Just pick some documentation from &amp;quot;industry leaders&amp;quot; like Langchain and see that not only is it already and always outdated, it sometimes simply contradicts itself.&lt;p&gt;In the &amp;quot;blockchain hype&amp;quot; this was similar, so I guess it&amp;#x27;s a trait of the hype train.</text></comment>
<story><title>Better and Faster Large Language Models via Multi-Token Prediction</title><url>https://arxiv.org/abs/2404.19737</url></story><parent_chain><item><author>deskamess</author><text>Side track: There is so much going on this space. I wish there was a chronological flow of a machine learning scenario&amp;#x2F;story with all the terms being introduced as we meet them (data, pre-training, training, inference, mixture of experts, RAG). Like someone walking me through a factory explaining what happens at each stage (like Mr Rogers used to do). Most of the time I do not know where the terms fit in the big picture. When I first came across pre-training I thought it was something done to the data before training happened but it was actually another training.</text></item></parent_chain><comment><author>highwaylights</author><text>I feel your pain and excitement in equal measure.&lt;p&gt;It can be hard to know where to start with some of these concepts, especially so given that a lot of recent developments (e.g. RAG) are developing so rapidly that there’s unlikely to be a reference book you could refer to anytime soon that would be current.&lt;p&gt;That said, I do find that documentation is getting better depending on where you look. The documentation for higher level tools like LlamaIndex is a good starting point for understanding the concepts (not so much in terms of &lt;i&gt;explaining&lt;/i&gt; the concepts, but showing where they fit into the overall picture, then you can deep-dive elsewhere on the different parts).&lt;p&gt;YouTube has always been a mixed bag of very little solid information in a sea of non-experts trying to attract clicks for the latest trends, so it’s not a great starting point IMHO.</text></comment>