It is also noteworthy that our LLaMA-2-70B model achieves competitive accuracy with early versions of GPT-4 on GSM8K and MATH. To our knowledge, this is the first LLaMA-based model to outperform GPT-4-0314 on MATH.