The results on LongBench (Bai et al., 2023) are presented in Table 12. This benchmark is specifically designed to test the long context ability of LLMs. We test the Chinese subsets of LongBench (including code tasks). The models marked with 16K were finetuned using Positional Interpolation (PI) method (Chen et al., 2023), which supports 16K context. The models marked with 64K were finetuned using YaRN method (Peng et al., 2023), which supports 64K context.