To examine the ability of tool manipulation, existing work mostly adopts complex reasoning tasks for evaluation, such as mathematical problem solving (_e.g._, GSM8k [184] and SVAMP [592]) or knowledge question answering (_e.g._, TruthfulQA [556]), where the successful utilization of tools is very important for enhancing the required skills that LLMs are incapable in (_e.g._, numerical calculation). In this way, the evaluated performance on these tasks can reflect the ability of LLMs in tool manipulation. To teach LLMs to utilize tools, existing studies add exemplars using tools in context to elicit LLMs [443], or fine-tune LLMs on simulated data about tool utilization [693, 80]. It has been found that with the help of tools, LLMs become more capable of handling the issues that they are not good at, _e.g._, equation calculation and answering timely questions [80, 448]. However, as the number of available tools increases, the limited context length of LLMs may pose challenges in describing and demonstrating extensive tool APIs. To address this issue, existing work retrieves the usage of relevant tools, or encoding tool information as tokens within the embedding space [702, 703, 704].