Recent progress on physics-based character animation has shown impressive breakthroughs on human motion synthesis, through imitating motion capture data via deep reinforcement learning. However, results have mostly been demonstrated on imitating a single distinct motion pattern, and do not generalize to interactive tasks that require flexible motion patterns due to varying human-object spatial configurations. To bridge this gap, we focus on one class of interactive tasks---sitting onto a chair. We propose a hierarchical reinforcement learning framework which relies on a collection of subtask controllers trained to imitate simple, reusable mocap motions, and a meta controller trained to execute the subtasks properly to complete the main task. We experimentally demonstrate the strength of our approach over different single level and hierarchical baselines. We also show that our approach can be applied to motion prediction given an image input. A video highlight can be found at https://youtu.be/XWU3wzz1ip8/.
 The capability of synthesizing realistic human-scene interactions is an important basis for simulating human living space, where robots can be trained to collaborate with humans, e.g. avoiding collisions or expediting the completion of assistive tasks. Motion capture (mocap) data, by offering high quality recordings of articulated human pose, has provided a crucial resource for human motion synthesis. With large mocap datasets and deep learning algorithms, kinematics-based approaches have recently made rapid progress on motion synthesis and prediction (Fragkiadaki et al., 2015; Jain et al., 2016; Holden et al., 2016; Ghosh et al., 2017; Bütepage et al., 2017; Martinez et al., 2017; Holden et al., 2017; Zhou et al., 2018; Gui et al., 2018a; b; Yan et al., 2018) . However, the lack of physical interpretability in their synthesized motion has been a major limitation of these approaches. The problem becomes especially clear when it comes to motions that involve substantial human-object or human-human interactions. Without modeling the physics, the sythensized interactions are often physically unrealistic, e.g. body parts penetrating obstacles or not reacting to collision. This generally limits the use of these approaches to either non-interactive motions, or a carefully set up virtual scene with high fidelity to the captured one. The graphics community has recently witnessed impressive progress on physics-based character animation (Peng et al., 2017; b) . These approaches, through imitating mocap examples via deep reinforcement learning, can synthesize realistic motions in physics simulated environments. Consequently, they can adapt to different physical contexts and thus attain a better generalization performance for interaction-based motions, e.g. walking on uneven terrain or stunt performance under obstacle disturbance. Nonetheless, these approaches still suffer from a drawback-a single model is trained for performing a single task with a distinct motion pattern (often time from a single mocap clip). As a result, they might not generalize to higher-level interactive tasks that require flexible motion patterns. Take the example of a person sitting down on a chair. A person can start in any location and orientation relative to the chair (Fig. 1) . A fixed motion pattern (e.g. turn left and sit) will be incapable of handling such variations. In this paper, we focus on one class of high-level interactive tasks-sitting onto a chair. As earlier mentioned, there are many possible human-chair configurations and different configurations may require different sequences of actions to accomplish the goal. For example, if the human is facing the chair, it needs to walk, turn either left or right, and sit; if the human is behind the chair, it needs to walk, side-walk and sit. To this end, we propose a hierarchical reinforcement learning (RL) method to address the challenge of generalization. Our key idea is the use of hierarchical control: (1) we assume the main task (e.g. sitting onto a chair) can be decomposed into several subtasks (e.g. walk, turn, sit, etc.), where the motion of each subtask can be reliably learned from mocap data, and (2) we train a meta controller using RL which can execute the subtasks properly to "complete" the main task from a given configuration. Such strategy is in line with the observation that humans have a repertoire of motion skills, and different subset of skills is selected and executed for different high-level tasks. Our contributions are three folds: (1) we extend the prior work on physics-based motion imitation to the context of higher-level interactive tasks using a hierarchical approach; (2) we experimentally demonstrate the strength of our hierarchical approach over different single level and hierarchical baselines; (3) we show at the end that our approach can be applied to motion synthesis in human living space with the help of 3D scene reconstruction.
Synthesizing human motions on interactive tasks using mocap data and hierarchical RL.
{"src": "scitldr-AIC_train", "paper_id": "HylvlaVtwr"}