Figure 1: The orange star markers represent the accuracy achieved by selecting the best response from 256 random generations of the LLaMA-2 7B model. The high accuracy on the MATH (left) and GSM8K (right) benchmarks (72.0% and 97.7%, respectively) suggest that the LLaMA-2 7B already possesses strong mathematical capabilities, although the stability in generating correct answers could be enhanced. This paper demonstrates that by scaling synthetic SFT data, the stability can be significantly improved as evidenced by the curves. Through this straightforward scaling of SFT data, the top-performing model has exceeded an early GPT-4 model by 10.3% on the MATH benchmark.