We introduceLumos, a novel framework for traininglanguage agents that employs aunified data format and amodular architecture based on open-source large language models (LLMs) .Lumosconsists of three distinct modules: planning, grounding, and execution. Theplanning modulebreaks down a task into a series of high-level, tool-agnostic subgoals, which are then made specific by thegrounding modulethrough a set of low-level actions. These actions are subsequently executed by theexecution module, utilizing a range of off-the-shelf tools and APIs. In order to train these modules effectively, high-quality annotations of subgoals and actions were collected and are made available for fine-tuning open-source LLMs for various tasks such as complex question answering, web tasks, and math problems. Leveraging this unified data and modular design,Lumosnot only achieves comparable or superior performance to current, state-of-the-art agents, but also exhibits several key advantages: (1) Lumossurpasses GPT-4/3.5-based agents in complex question answering and web tasks, while equalling the performance of significantly larger LLM agents on math tasks; (2) Lumosoutperforms open-source agents created through conventional training methods and those using chain-of-thoughts training; and (3) Lumosis capable of effectively generalizing to unseen interactive tasks, outperforming larger LLM-based agents and even exceeding performance of specialized agents.