Previous studies have demonstrated that continual pre-training can significantly improve the model's ability to understand and generate specific content Gupta et al. (2023). However, continual pre-training could lead to catastrophic forgetting French (1999), and limited research has explored the abilities forgotten during pre-training on an existing fine-tuned LLM.