We extend the learning from demonstration paradigm by providing a method for learning unknown constraints shared across tasks, using demonstrations of the tasks, their cost functions, and knowledge of the system dynamics and control constraints. Given safe demonstrations, our method uses hit-and-run sampling to obtain lower cost, and thus unsafe, trajectories. Both safe and unsafe trajectories are used to obtain a consistent representation of the unsafe set via solving a mixed integer program. Additionally, by leveraging a known parameterization of the constraint, we modify our method to learn parametric constraints in high dimensions. We show that our method can learn a six-dimensional pose constraint for a 7-DOF robot arm. Inverse optimal control and inverse reinforcement learning (IOC/IRL) BID5 can enable robots to perform complex goaldirected tasks by learning a cost function which replicates the behavior of an expert demonstrator when optimized. However, planning for many robotics and automation tasks also requires knowing constraints, which define what states or trajectories are safe. Existing methods learn local trajectory-based constraints BID3 BID4 or a cost penalty to approximate a constraint BID1 , neither of which extracts states that are guaranteed unsafe for all trajectories. In contrast, recent work BID2 recovers a binary representation of globally-valid constraints from expert demonstrations by sampling lower cost (and hence constraintviolating) trajectories and then recovering a constraint consistent with the data by solving an integer program over a gridded constraint space. The learned constraint can be then used to inform a planner to generate safe trajectories connecting novel start and goal states. However, the gridding restricts the scalability of this method to higher dimensional constraints. The contributions of this workshop paper are twofold:• By assuming a known parameterization of the constraint, we extend BID2 to higher dimensions by writing a mixed integer program over parameters which recovers a constraint consistent with the data.• We evaluate the method by learning a 6-dimensional pose constraint on a 7 degree-of-freedom (DOF) robot arm. II . PRELIMINARIES AND PROBLEM STATEMENT We consider a state-control demonstration (ξ * x . = {x 0 , . . . , x T }, ξ * u . = {u 0 , . . . , u T −1 }) which steers a controlconstrained system x t+1 = f (x t , u t , t), u t ∈ U for all t, from a start state x 0 to a goal state x T , while minimizing cost c(ξ x , ξ u ) and obeying safety constraints φ(ξ) . DISPLAYFORM0 Formally, a demonstration solves the following problem 1 : DISPLAYFORM1 are known functions mapping (ξ x , ξ u ) to some constraint spaces C andC, where subsets S ⊆ C and S ⊆C are considered safe. In particular ,S is known and represents the set of all constraints known to the learner.In this paper, we consider the problem of learning the unsafe set A . DISPLAYFORM2 , each with different start and goal states. We assume that the dynamics, control constraints, and start and goal constraints are known and are embedded inφ(ξ x , ξ u ) ∈S. We also assume the cost function c(·, ·) is known. BID0 Details for continuous-time and suboptimal demonstrations are in BID2 . DISPLAYFORM3
We can learn high-dimensional constraints from demonstrations by sampling unsafe trajectories and leveraging a known constraint parameterization.
{"src": "scitldr-AIC_train", "paper_id": "SJxqVK-opV"}