Recent work has shown the immense potential of synthetically generated datasets for training large language models (LLMs) , especially for acquiring targeted skills.
Current large-scale math instruction tuning datasets such as MetaMathQAand MAmmoTHare constructed using outputs from closed-source LLMs with commercially restrictive licenses.
A key reason limiting the use of open-source LLMs in these data generation pipelines has been the wide gap between the mathematical skills of the best closed-source LLMs, such as GPT-4, and the best open-source LLMs.
Building on the recent progress in open-source LLMs, our proposed prompting novelty, and some brute-force scaling, we construct OpenMathInstruct-1, a math instruction tuning dataset with 1.8M problem-solution pairs.
The dataset is constructed by synthesizingcode-interpretersolutions for GSM8K and MATH, two popular math reasoning benchmarks, using the recently released and permissively licensed Mixtral model.
Our best model, OpenMath-CodeLlama-70B, trained on a subset of OpenMathInstruct-1, achieves a score of 84.6% on GSM8K and 50.7% on MATH, which is competitive with the bestgpt-distilledmodels.
We release our code, models, and the OpenMathInstruct-1 dataset under a commercially permissive license.111Data and models are available athttps://huggingface.co/collections/nvidia/openmath-65c5619de2ba059be0775014Code is available athttps://github.com/Kipok/NeMo-Skills