Hierarchical Reinforcement Learning for Quadruped Locomotion
Legged locomotion is a challenging task for learning algorithms, especially when the task requires a diverse set of primitive behaviors. To solve these problems, we introduce a hierarchical framework that can automatically learn to decompose complex locomotion tasks. A high level policy issues commands in the form of a latent vector and also selects for how long the low level policy will execute the latent command. Concurrently, the low level policy uses the latent command and only the robot's on board sensors to control the robot's actuators. Our approach allows the high level policy to run at a lower frequency than the low level one. We test our framework on a path following task for a dynamic quadruped robot and we show that steering behaviors automatically emerge in the latent command space as low level skills are needed for this task. We then show efficient adaptation of the trained policy to new tasks by transfer of the trained low level policy. Finally, we validate the policies on a real quadruped robot. To the best of our knowledge, this is the first application of end to end hierarchical learning to a real robotic locomotion task.
2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)