\(\bullet\)_Direct model fine-tuning._ To adapt LLMs to a long context window, a straightforward approach is to directly fine-tune the models on long texts with the desired length. The context extension can be scheduled with increased lengths in a multi-stage approach (_e.g._, \(2\text{K}\to 8\text{K}\to 32\text{K}\)). To conduct effective extension, it needs specially prepared long texts for training. Specially, some recent study has shown that the quality is more important than the lengths of training text in long context models [288]. However, a recent study has highlighted that the fine-tuning approach tends to be inherently slow when adapting LLMs for long texts [240].