Learning to plan in synthetic environments.An important feature of the agent in Autonomous Machine Intelligence (LeCun, 2022) is planning--an ability of decomposing complex tasks into subtasks and finishing the subtasks in a reward-optimal way (Kambhampati et al., 2024). Synthetic data can be a valuable tool here as it can serve as the feedback signal collected from a simulator (Park et al., 2023), and learning on it can make the agent aware of affordances (Ahn et al., 2022; Liang et al., 2022). For example, Inner Monologue (Huang et al., 2022) leverages natural language form feedback generated by the simulated environment to teach LLM-based robots planning. They find that such feedback significantly improves high-level instruction completion on both simulated and real-world domains. To compose a large number of realistic planning tasks (e.g., _"Rearrange objects on a table to match a given scene."_), VIMA (Jiang et al., 2022) creates a multi-modality simulated environment called VIMA-Bench, which supports extensible collections of objects and textures. In the Minecraft game, Voyager (Wang et al., 2023) deploys a number of GPT-4 based agents to interact with the synthetic environment and finds that the agents can unlock new skills faster and complete planning more efficiently with the help of synthetic feedback.