Solving Quantitative Reasoning Problems with
Language Models
Aitor Lewkowycz∗, Anders Andreassen†, David Dohany, Ethan Dyery, He
Solving Quantitative Reasoning Problems with Language Models
Language models have achieved remarkable performance on a wide range of tasks
that require natural language understanding. Nevertheless, state-of-the-art
models have generally struggled with tasks that require quantitative reasoning,
such as solving mathematics, science, and engineering problems at the college
level. To help close this gap, we introduce Minerva, a large language model
pretrained on general natural language data and further trained on technical
content. The model achieves state-of-the-art performance on technical
benchmarks without the use of external tools. We also evaluate our model on
over two hundred undergraduate-level problems in physics, biology, chemistry,
economics, and other sciences that require quantitative reasoning, and find
that the model can correctly answer nearly a third of them.
Aitor Lewkowycz Anders Andreassen David Dohan Ethan Dyer Henryk Michalewski Vinay Ramasesh Ambrose Slone Cem Anil Imanol Schlag Theo Gutman-Solo Yuhuai Wu Behnam Neyshabur Guy Gur-Ari Vedant Misra