**Reasoning.** The feedback data from the environment may not be directly suitable to be utilized by LLMs for plan refinement, _e.g.,_ containing irrelevant information or taking a non-language form. To solve this, some work adds the explicit reasoning process to extract critical information from feedback [448, 449]. For example, React [449] prompts LLMs with demonstrations to generate reasoning traces over feedback. It has been widely used in autonomous agent projects, such as AutoGPT [534], which can automatically reason over the observed feedback to revise the initial plan for solving various user requests. However, these approaches typically fix the order of reasoning and planning. To support flexible switching between the two processes for better performance, ChatCoT [448] further unifies the tool-augmented reasoning process into a multi-turn conversation between the LLM-based task planner and the tool-based environment.