Lastly, we consider the task of arithmetic reasoning. Existing work shows that LMs can struggle with evaluating arithmetic expressions correctly [1]}. While reasoning steps might be correct, mistakes in the concrete arithmetic calculations will lead to an incorrect result [1]}, [3]}. This is exacerbated by the open-ended nature of math problems, where the result is not picked from a limited set of options, but can be any valid number. Recent works [1]}, [3]}, [6]} therefore propose to augment LM generation with the ability to externally evaluate arithmetic expressions on-the-fly.
<TABLE><FIGURE>