The straightforward synthetic SFT data proves effective from stronger base models as well, such as LLaMA-2 7OB, which achieves 90.6% on GSM8K and 52.8% on MATH. To the best of our knowledge, this is the first open-source model to exceed 90% accuracy on GSM8K. It is also the first open-source model to outperform GPT-4 (i.e., GPT-4-0314) on the MATH benchmark, demonstrating the efficacy of our simple synthetic scaling method.