Another issue of interest to us is whether the improvement in Chinese proficiency has an impact on the existing English capabilities. To address this question, we additionally collected 200,000 Chinese samples from the internet and randomly extracted 200,000 English samples from the refinedweb dataset [20]. Utilizing these samples, we evaluate the English perplexity and Chinese perplexity of LLaMA models trained on different-scale corpora, as depicted in table 2. Our findings reveal that with the increase in further pretraining scale, the perplexity of the models decreases steadily in Chinese, yet notably increases in English. This suggests that enhancing the model's capabilities solely through a single Chinese corpus comes at the cost of sacrificing the original English proficiency.