**Scaling Position Embeddings.** Transformer-based LLMs can learn effective position embeddings within the maximum training length. Thus, when adapting LLMs to language tasks beyond the maximum training length, it is necessary to scale to larger position indices. Some specific position embeddings have been shown to possess a certain degree of ability to generalize to text beyond the training length, which is formally termed _extrapolation capability_, including T5 bias [82], ALiBi [264], xPos [277] and even NoPE [287]. However, as one of the mainstream position embedding methods, RoPE exhibits limited extrapolation ability in empirical studies [240]. In the following, we discuss several methods that can scale RoPE to longer texts.