We have noticed that many advancements in fine-tuned variants adhere to a conventional procedure consisting of two key steps: 1. Conduct further continual pre-training on an existing LLM. 2. Carry out subsequent alignment operations, such as Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF), on the model obtained in Step 1. Among these fine-tuned variants, many developments perform further continual pre-training on existing fine-tuned LLMs Cui et al. (2023); Lin and Chen (2023).