@inproceedings{
dalal2023planseqlearn,
title={Plan-Seq-Learn: Language Model Guided {RL} for Solving Long Horizon Robotics Tasks},
author={Murtaza Dalal and Tarun Chiruvolu and Devendra Singh Chaplot and Ruslan Salakhutdinov},
booktitle={NeurIPS 2023 Foundation Models for Decision Making Workshop},
year={2023},
url={https://openreview.net/forum?id=VIxIiabqWj}
}
Large Language Models (LLMs) have been shown to be capable of performing high-level planning for long-horizon robotics tasks, yet existing methods require access to a pre-defined skill library (_e.g._ picking, placing, pulling, pushing, navigating).
However, LLM planning does not address how to design or learn those behaviors, which remains challenging particularly in long-horizon settings. Furthermore, for many tasks of interest, the robot needs to be able to adjust its behavior in a fine-grained manner, requiring the agent to be capable of modifying _low-level_ control actions. Can we instead use the internet-scale knowledge from LLMs for high-level policies, guiding reinforcement learning (RL) policies to efficiently solve robotic control tasks online without requiring a pre-determined set of skills? In this paper, we propose *Plan-Seq-Learn* (PSL): a modular approach that uses motion planning to bridge the gap between abstract language and learned low-level control for solving long-horizon robotics tasks from scratch. We demonstrate that PSL is capable of solving 25+ challenging single and multi-stage robotics tasks on four benchmarks at success rates of over 85\% from raw visual input, out-performing language-based, classical, and end-to-end approaches. 
Video results and code at https://mihdalal.github.io/planseqlearn/
Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Tasks
Murtaza Dalal Tarun Chiruvolu Devendra Singh Chaplot Ruslan Salakhutdinov