**Evaluation.** We evaluate MiniCPM-2.4B-128K in \(\infty\)Bench (Zhang et al., 2024b), a pioneering benchmark for long context evaluations. The tasks in \(\infty\)Bench (Zhang et al., 2024b) extend beyond typical retrieval tasks and challenge the model with long context reasoning. We can see in Table 5, we achieve comparable results in Mistral-7B-Instruct-v0.2 (ABF1000w) and outperform ChatGLM3-6B-128K despite being 2.5 times smaller.