<story><title>Lossless Acceleration of LLM via Adaptive N-Gram Parallel Decoding</title><url>https://arxiv.org/abs/2404.08698</url></story><parent_chain></parent_chain><comment><author>kristjansson</author><text>This is speculative decoding with a n-gram Markov chain instead of a weaker transformer model in the “speculating” position.</text></comment>
<story><title>Lossless Acceleration of LLM via Adaptive N-Gram Parallel Decoding</title><url>https://arxiv.org/abs/2404.08698</url></story><parent_chain></parent_chain><comment><author>nsagent</author><text>How does this differ from the 2018 NeurIPS paper, Blockwise Parallel Decoding for Deep Autoregressive Models?&lt;p&gt;&lt;a href=&quot;https:&amp;#x2F;&amp;#x2F;arxiv.org&amp;#x2F;abs&amp;#x2F;1811.03115&quot; rel=&quot;nofollow&quot;&gt;https:&amp;#x2F;&amp;#x2F;arxiv.org&amp;#x2F;abs&amp;#x2F;1811.03115&lt;/a&gt;</text></comment>