\(\bullet\)_Decoding strategy optimization_ aims to improve the sequential nature of the auto-regressive generation manner in different ways. As a representative study, _speculative decoding_[323, 324] first leverages a compact but efficient model (_e.g._, a \(n\)-gram model or a small PLM) to generate short segments and then utilizes the LLM to verify and correct these drafts. It can lead to a notable \(2\times\) to \(3\times\) speedup without compromising the generation quality. Researchers further suggest several variants to improve the efficiency of this approach, such as a learning-based method to combine several small models [325] and a stage-wise acceleration which employs a more smaller LM to accelerate the small LM first [326]. In addition, token-level early-exit techniques have been proposed enabling the generation of a token at lower Transformer layers, rather than passing through all the layers [327]. It can attain greater speedup, but at the cost of sacrificing generation quality.