@inproceedings{
dugan2024occamllm,
title={Occam{LLM}: Fast and Exact Language Model Arithmetic in a Single Step},
author={Owen M Dugan and Donato M. Jim{\'e}nez Benet{\'o} and Charlotte Loh and Zhuo Chen and Rumen Dangovski and Marin Soljacic},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=vAOgaPvgYr}
}
Despite significant advancements in text generation and reasoning, Large Language Models (LLMs) still face challenges in accurately performing complex arithmetic operations. Language model systems often enable LLMs to generate code for arithmetic operations to achieve accurate calculations. However, this approach compromises speed and security, and fine-tuning risks the language model losing prior capabilities. We propose a framework that enables exact arithmetic in *a single autoregressive step*, providing faster, more secure, and more interpretable LLM systems with arithmetic capabilities. We use the hidden states of a LLM to control a symbolic architecture that performs arithmetic. Our implementation using Llama 3 with OccamNet as a symbolic model (OccamLlama) achieves 100\% accuracy on single arithmetic operations ($+,-,\times,\div,\sin{},\cos{},\log{},\exp{},\sqrt{}$), outperforming GPT 4o with and without a code interpreter. Furthermore, OccamLlama outperforms GPT 4o with and without a code interpreter on average across a range of mathematical problem solving benchmarks, demonstrating that OccamLLMs can excel in arithmetic tasks, even surpassing much larger models. Code is available at https://github.com/druidowm/OccamLLM.
OccamLLM: Fast and Exact Language Model Arithmetic in a Single Step
Owen M Dugan Donato M. Jiménez Benetó Charlotte Loh Zhuo Chen Rumen Dangovski Marin Soljacic