We present a framework for learning to plan hierarchically in domains with unknown dynamics. We enhance planning performance by exploiting problem structure in several ways: First, we simplify the search over plans by leveraging knowledge of skill objectives; second, shorter plans are generated by enforcing aggressively hierarchical planning; and third, we learn transition dynamics with sparse local models for better generalization. Our framework decomposes transition dynamics into skill effects and success conditions, which allows fast planning by reasoning on effects, while learning conditions from interactions with the world. We propose a simple method for learning new abstract skills, using successful trajectories stemming from completing the goals of a curriculum. Learned skills are then refined to leverage other abstract skills and enhance subsequent planning. We show that both conditions and abstract skills can be learned simultaneously while planning, even in stochastic domains. Our method is validated in experiments of increasing complexity, with up to @math states, showing superior planning to classic non-hierarchical planners or reinforcement learning methods. Applicability to real-world problems is demonstrated in a simulation-to-real transfer experiment on a robotic manipulator.
The idea of planning by reasoning about action effects and conditions has been long studied, and is a core principle behind classic planners like STRIPS @cite_18 . These planners typically require all action effects and conditions to be specified, and often produce sequential plans, thus lacking the advantages of hierarchical planning. Hierarchical Task Networks (HTN) @cite_31 provide a hierarchical alternative by producing plans given skills and their dependencies. HTN planners, and their extension to AND OR graphs @cite_6 , have been successfully applied to rich and complex problems such as robot soccer @cite_5 , multi-agent assembly domains @cite_35 , and human-robot collaborative assembly @cite_25 . However, these planners require dynamics and hierarchy to be specified.
@cite_35 @cite_18 @cite_6 @cite_5 @cite_31 @cite_25
2129975315 2115181241 2521107005 1748530542
In this paper we address the problem of generating preferred plans by combining the procedural control knowledge specified by Hierarchical Task Networks (HTNs) with rich user preferences. To this end, we extend the popular Planning Domain Definition Language, PDDL3, to support specification of simple and temporally extended preferences over HTN constructs. To compute preferred HTN plans, we propose a branch-and-bound algorithm, together with a set of heuristics that, leveraging HTN structure, measure progress towards satisfaction of preferences. Our preference-based planner, HTNPLAN-P, is implemented as an extension of the SHOP2 planner. We compared our planner with SGPlan5 and HPLAN-P- the top performers in the 2006 International Planning Competition preference tracks. HTNPLAN-P generated plans that in all but a few cases equalled or exceeded the quality of plans returned by HPLAN-P and SGPlan5. While our implementation builds on SHOP2, the language and techniques proposed here are relevant to a broad range of HTN planners. A key feature of modern optimal planners such as graphplan and blackbox is their ability to prune large parts of the search space. Previous Partial Order Causal Link (POCL) planners provide an alternative branching scheme but lacking comparable pruning mechanisms do not perform as well. In this paper, a domain-independent formulation of temporal planning based on Constraint Programming is introduced that successfully combines a POCL branching scheme with powerful and sound pruning rules. The key novelty in the formulation is the ability to reason about supports, precedences, and causal links involving actions that are not in the plan. Experiments over a wide range of benchmarks show that the resulting optimal temporal planner is much faster than current ones and is competitive with the best parallel planners in the special case in which actions have all the same duration. We present an approach to coordinate the behavior of a multiagent team using an HTN planning procedure. To coordinate teams, high level tasks have to be broken down into subtasks which is a basic operation in HTN planners. We are using planners in each of the agents to incorporate domain knowledge and to make agents follow a specified team strategy. With our approach, agents coordinate deliberatively and still maintain a high degree of reactivity. In our implementation for use in RoboCup Simulation League, first results were already very promising. Using a planner leads to better separation of agent code and expert knowledge. In the last several years the computational complexity of classical planning and HTN planning have been studied. But in both cases it is assumed that the planner has complete knowledge about the initial state. Recently, there has been proposal to use 'sensing' actions to plan in presence of incompleteness. In this paper we study the complexity of planning in such cases. In our study we use the action description language A proposed in 1993 by M. Gelfond and V. Lifschitz and its extensions. The language A allows planning in the situations with complete information. It is known that, if we consider only plans of feasible (polynomial) length, the planning problem for such situations is NP-complete: even checking whether a given objective is attainable from a given initial state is NP-complete. In this paper, we show that the planning problem in presence of incompleteness is indeed harder: it belongs to the next level of complexity hierarchy (in precise terms, it is Σ2P-complete). To overcome the complexity of this problem, C. Baral and T. Son have proposed several approximations. We show that under certain conditions, one of these approximations - O-approximation - makes the problem NP-complete (thus indeed reducing its complexity).