(Bai et al., 2022; Ouyang et al., 2022)), the model can correctly solve 97.7% of GSM8K questions and 72.0% of MATH questions, when selecting the best answer from 256 random generations, as indicated by the orange star marks in Figure 1. It is noteworthy that the accuracy has even outperformed those reported for the GPT-4 model, which achieved 92.0% on GSM8K and 42.5% on MATH 1. Therefore, we conclude that the LLaMA-2 7B model has indeed developed strong mathematical capabilities. The primary issue is the lack of guarantee that the correct answer will be digged out, as most generations are incorrect. In fact, the accuracy drops to 49.5% on GSM8K and 7.9% on MATH if we consider only one random generation per question. We refer to this as the _instability issue_.