We present a framework for learning to plan hierarchically in domains with unknown dynamics. We enhance planning performance by exploiting problem structure in several ways: First, we simplify the search over plans by leveraging knowledge of skill objectives; second, shorter plans are generated by enforcing aggressively hierarchical planning; and third, we learn transition dynamics with sparse local models for better generalization. Our framework decomposes transition dynamics into skill effects and success conditions, which allows fast planning by reasoning on effects, while learning conditions from interactions with the world. We propose a simple method for learning new abstract skills, using successful trajectories stemming from completing the goals of a curriculum. Learned skills are then refined to leverage other abstract skills and enhance subsequent planning. We show that both conditions and abstract skills can be learned simultaneously while planning, even in stochastic domains. Our method is validated in experiments of increasing complexity, with up to @math states, showing superior planning to classic non-hierarchical planners or reinforcement learning methods. Applicability to real-world problems is demonstrated in a simulation-to-real transfer experiment on a robotic manipulator.
Learning the skill hierarchy while planning with HTN is proposed in @cite_24 , but relies on expert demonstrations, and transition dynamics still need to be specified. More recent work requires a graphical task representation to automatically construct HTMs @cite_7 . These techniques necessitate considerable expert knowledge. By opposition, our work aims to learn both dynamics and hierarchy directly from interactions with the world.
@cite_24 @cite_7
2162227979 2409715576
Knowledge-based planning methods offer benefits over classical techniques, but they are time consuming and costly to construct. There has been research on learning plan knowledge from search, but this can take substantial computer time and may even fail to find solutions on complex tasks. Here we describe another approach that observes sequences of operators taken from expert solutions to problems and learns hierarchical task networks from them. The method has similarities to previous algorithms for explanation-based learning, but differs in its ability to acquire hierarchical structures and in the generality of learned conditions. These increase the method's capability to transfer learned knowledge to other problems and supports the acquisition of recursive procedures. After presenting the learning algorithm, we report experiments that compare its abilities to other techniques on two planning domains. In closing, we review related work and directions for future research. Collaboration between humans and robots requires solutions to an array of challenging problems, including multi-agent planning, state estimation, and goal inference. There already exist feasible solutions for many of these challenges, but they depend upon having rich task models. In this work we detail a novel type of Hierarchical Task Network we call a Clique Chain HTN (CC-HTN), alongside an algorithm for autonomously constructing them from topological properties derived from graphical task representations. As the presented method relies on the structure of the task itself, our work imposes no particular type of symbolic insight into motor primitives or environmental representation, making it applicable to a wide variety of use cases critical to human-robot interaction. We present evaluations within a multi-resolution goal inference task and a transfer learning application showing the utility of our approach.