Recent large language models (LLMs)  advancements sparked a growing research interest in tool assisted LLMs solving real-world challenges, which calls for comprehensive evaluation of tool-use capabilities. While previous works focused on either evaluating over stateless web services (RESTful API) , based on a single turn user prompt, or an off-policy dialog trajectory,ToolSandbox111ToolSandboxevaluation framework is released athttps://github.com/apple/ToolSandboxincludes stateful tool execution, implicit state dependencies between tools, a built-in user simulator supporting on-policy conversational evaluation and a dynamic evaluation strategy for intermediate and final milestones over an arbitrary trajectory. We show that open source and proprietary models have a significant performance gap, and complex tasks like State Dependency, Canonicalization and Insufficient Information defined inToolSandboxare challenging even the most capable SOTA LLMs, providing brand-new insights into tool-use LLM capabilities.