**Adapting Context Window.** Since Transformer-based LLMs have limited context windows, they can not directly integrate or utilize the entire information of the long sequences exceeding the context window. To alleviate the limitation, several methods adapting LLMs to long context have been proposed, as discussed below.