We performed a lightweight continual pre-training for long-context adaptation of existing LLMs. The pre-training data includes the PG19 and Arxiv-math corpus as well as C4 text (Raffel et al., 2020) with length more than 4K tokens. The segment length \(N\) was set to 2K throughout our experiments.