@inproceedings{
chen2023tooldec,
title={ToolDec: Syntax Error-Free and Generalizable Tool Use for {LLM}s via Finite-State Decoding},
author={Hongqiao Chen and Kexun Zhang and Lei Li and William Yang Wang},
booktitle={The 3rd Workshop on Mathematical Reasoning and AI at NeurIPS'23},
year={2023},
url={https://openreview.net/forum?id=gUJLxVpWi2}
}
Large language models (LLMs) have shown promising capabilities in using external tools.
However, existing approaches rely on fine-tuning or in-context learning to use tools, which make syntactic mistakes and are difficult to generalize.
In this paper, we propose ToolDec, a finite-state machine-guided decoding algorithm for tool-augmented LLMs.
ToolDec eliminates tool-related errors by ensuring valid tool names and type-conforming arguments.
Furthermore, ToolDec enables LLM to effectively select tools using only the information contained in their names, with no need for tool-specific fine-tuning.
Our experiments on multiple word problem datasets show that ToolDec reduces syntactic errors to zero, consequently achieving significantly better performance and as much as a 2x speedup.
We also show that ToolDec achieves superior generalization performance on unseen tools, performing up to 8x better than the baseline.
ToolDec: Syntax Error-Free and Generalizable Tool Use for LLMs via Finite-State Decoding
Hongqiao Chen Kexun Zhang Lei Li William Yang Wang