* **Resoning Irconsistency.** LLMs may generate the correct answer following an invalid reasoning path, or produce a wrong answer after a correct reasoning process, leading to inconsistency between the derived answer and the reasoning process. The issue can be alleviated by fine-tuning LLMs with process-level feedback, using an ensemble of diverse reasoning paths, and refining the reasoning process with self-reflection or external feedback. * **Numerical computation.** For complex reasoning tasks, LLMs still face difficulties in the involved numerical computation, especially for the symbols that are seldom encountered during pre-training, such as arithmetic with large numbers [677, 690, 49]. To tackle this issue, a direct way is to tune LLMs on synthesized arithmetic problems [691, 361]. Also, a surge of studies improve the numerical computation performance by tracing intermediate calculation steps in training and inference stages [676, 692, 361], _e.g.,_ scratched tracing. In addition, existing work [80] has also incorporated external tools (_e.g.,_ calculator), especially for handling arithmetic operations. More recently, ChatGPT has provided a plugin mechanism to use external tools [665]. In this way, LLMs need to learn how to properly manipulate the tools. For this purpose, researchers have augmented the examples using tools (even the LLM itself) for tuning the LLM [80, 693], or devised instructions and exemplars for in-context learning [443]. In addition to the aid of external tools, recent studies find that tokenizing digits into individual tokens (_e.g.,_ LLMA and Galactica tokenizers) is a useful approach to enhancing the inherent arithmetic ability of LLMs [690, 361]. One possible explanation is that subword tokenization techniques can result in inconsistent sequences when tokenizing numbers. For instance, with a subword tokenizer the integer 7481 may be tokenized as \(7\_481\), while 74815 may be tokenized as \(748\_15\) (the same numerical substrings with different splits) [361]. As a comparison, digit-based tokenization for numbers can avoid such an inconsistency, thus likely improving the numerical computation ability of LLMs.
* **Numerical Computation.** LLMs face difficulties in numerical computation, especially for the symbols that are seldom encountered during pre-training. In addition to using mathematical tools, tokenizing digits into individual tokens is also an effective design choice for improving the arithmetic ability of LLMs.