@inproceedings{
liu2023tinygsm,
title={Tiny{GSM}: achieving 80\% on {GSM}8k with one billion parameters},
author={Bingbin Liu and Sebastien Bubeck and Ronen Eldan and Janardhan Kulkarni and Yuanzhi Li and Anh Nguyen and Rachel Ward and Yi Zhang},
booktitle={The 3rd Workshop on Mathematical Reasoning and AI at NeurIPS'23},
year={2023},
url={https://openreview.net/forum?id=ROOVUBZp8v}
}
Small models offer various computational advantages, yet the extent to which size is critical for problem-solving abilities remains an open question. This work studies the performance of small models on mathematical reasoning. Specifically, for solving math word problems, we find that a 1.3B model can achieve 80.1% accuracy on GSM8K, outperforming existing models that are orders of magnitude larger, and even rivaling the performance of the GPT-3.5-turbo teacher model from which the training data is generated. Our approach is simple and has two key components: The first is the use of a GPT-3.5-turbo-generated synthetic dataset of math word problem with solutions, which we will fully release. The second component is the use of a verifier, which selects the final outputs from multiple candidate generations.
TinyGSM: achieving 80
Bingbin Liu Sebastien Bubeck Ronen Eldan Janardhan Kulkarni Yuanzhi Li Anh Nguyen Rachel Ward Yi Zhang