Large Language ModelsLarge language models Brown et al. (2020); Achiam et al. (2023); Touvron et al. (2023, 2023) have made significant achievements, with impressive performance on a wide range of tasks. Currently, closed-source large language models, represented by GPT Brown et al. (2020); Achiam et al. (2023), Gemini Team et al. (2023), Groke (xAI, 2023), and Claude-2 (Anthropic, 2023), are the most advanced models in terms of performance. However, open-source models, represented by LLaMA Touvron et al. (2023), LLaMA-2 Touvron et al. (2023) and Mixtral Jiang et al. (2024), have also progressed rapidly, and have even shown competitive performance with the closed-source models on some tasks. Our work, which aims to improve the performance of open-source LLMs on mathematical tasks by fine-tuning them on synthetic data.