On LLaMA-2-7B, our approach exceeds the prior best by absolutely +14.2 on GSM8K (compared to MuggleMath-7B (Li et al., 2023)), and by +20.8 on MATH (compared to MetaMath-7B (Yu et al., 2023)), respectively. It even surpasses several latest 70B models dedicated for math capabilities, such as WizardMath-70B (Luo et al., 2023) (82.6 versus 81.6 on GSM8K). On LLaMA-2-13B, the improvements are +14.1 on GSM8K (compared to MuggleMath-13B (Li et al., 2023)) and +22.5 on MATH (compared to MetaMath-13B (Yu et al., 2023)), respectively. On LLaMA-2-70B, the gains are +7.1 on GSM8K (compared to LEMA-LLaMA-2-70B (An et al., 2023)) and +26.2 on MATH (compared to MetaMath-70B (Yu et al., 2023)), respectively.