We describe a whole-body dynamic walking controller implemented as a convex quadratic program. The controller solves an optimal control problem using an approximate value function derived from a simple walking model while respecting the dynamic, input, and contact constraints of the full robot dynamics. By exploiting sparsity and temporal structure in the optimization with a custom active-set algorithm, we surpass the performance of the best available off-the-shelf solvers and achieve 1kHz control rates for a 34-DOF humanoid. We describe applications to balancing and walking tasks using the simulated Atlas robot in the DARPA Virtual Robotics Challenge.
Other uses of active-set methods for MPC have exploited the temporal relationship between the QPs arising in MPC. compared active-set and interior-point strategies for MPC @cite_22 . The described an active-set approach based on Schur complements for efficiently resolving KKT conditions after changes are made to the active set. This framework is analogous to the solution method we discuss in Section . In the discrete time setting, Wang and Boyd @cite_13 describe an approach to quickly evaluating control-Lyapunov policies using explicit enumeration of active sets in cases where the number of states is roughly equal to the square of the number of inputs.
@cite_13 @cite_22
2095633277 1983864916 1977192803 2963414638
We consider a comparison of active set vs. interior point strategies for the solution of receding time horizon problems in nonlinear model predictive control (NMPC). For this study we consider a control algorithm where we form quadratic programs (QPs) in each time horizon by linearizing the model. We also ignore second order information on the model and constraints. This approach can be viewed as a direct nonlinear extension of MPC with linear models and is easily tailored to include stabilizing constraints. Using this framework we consider the application of three active set strategies as well as interior point methods applied to both the NMPC and the QP subproblem. The first two active set methods (QPOPT and and QKWIK) are general purpose solvers that have been incorporated into SQP algorithms previously, while the third is a Schur complement approach that can easily exploit the sparse structure of the KKT matrix in MPC. Nearly all algorithms for linear model predictive control (MPC) either rely on the solution of convex quadratic programs (QPs) in real time, or on an explicit precalculation of this solution for all possible problem instances. In this paper, we present an online active set strategy for the fast solution of parametric QPs arising in MPC. This strategy exploits solution information of the previous QP under the assumption that the active set does not change much from one QP to the next. Furthermore, we present a modification where the CPU time is limited in order to make it suitable for strict real-time applications. Its performance is demonstrated with a challenging test example comprising 240 variables and 1191 inequalities, which depends on 57 parameters and is prohibitive for explicit MPC approaches. In this example, our strategy allows CPU times of well below 100 ms per QP and was about one order of magnitude faster than a standard active set QP solver. Copyright © 2007 John Wiley & Sons, Ltd. The evaluation of a control-Lyapunov policy, with quadratic Lyapunov function, requires the solution of a quadratic program (QP) at each time step. For small problems this QP can be solved explicitly; for larger problems an online optimization method can be used. For this reason the control-Lyapunov control policy is considered a computationally intensive control law, as opposed to an “analytical” control law, such as conventional linear state feedback, linear quadratic Gaussian control, or H∞, too complex or slow to be used in high speed control applications. In this note we show that by precomputing certain quantities, the control-Lyapunov policy can be evaluated extremely efficiently. We will show that when the number of inputs is on the order of the square-root of the state dimension, the cost of evaluating a control-Lyapunov policy is on the same order as the cost of evaluating a simple linear state feedback policy, and less (in order) than the cost of updating a Kalman filter state estimate. To give an idea of the speeds involved, for a problem with 100 states and 10 inputs, the control-Lyapunov policy can be evaluated in around 67 μs, on a 2 GHz AMD processor; the same processor requires 40 μs to carry out a Kalman filter update. In this paper we present foundations for using model predictive control (MPC) as a differentiable policy class in reinforcement learning. Specifically, we differentiate through MPC by using the KKT conditions of the convex approximation at a fixed point of the solver. Using this strategy, we are able to learn the cost and dynamics of a controller via end-to-end learning in a larger system. We empirically show results in an imitation learning setting, demonstrating that we can recover the underlying dynamics and cost more efficiently and reliably than with a generic neural network policy class.