\(\bullet\)_Base modification._ LLMs are usually trained with a pre-set maximum training length, _e.g._, 4096 in Llama 2 [99]. However, wavelengths in certain dimensions of RoPE may exceed the training length for longer text [276], so that language models have not undergone sufficient training (_i.e._, a complete rotation cycle) on these dimensions. Thus, when we adapt LLMs to longer texts, the rotation angles for certain dimensions would be never seen in the training phase [292]. Given a fixed rotation angle \(t\cdot\theta_{i}\), a smaller basis \(\theta_{i}\) allows for a greater distance \(t\), _i.e._, enabling the modeling of longer texts [235, 276, 288]. According to the formula \(\theta_{i}=b^{-2(i-1)/d}\) in Equation 4, decreasing the basis can be achieved by increasing the value of the base. In addition, decreasing the base can also help re-scale the wavelengths of all dimensions below the training length, while it often needs continual pre-training to adapt the LLMs to long context windows [292].