In this paper, we introduce Path Integral Networks (PI-Net), a recurrent network representation of the Path Integral optimal control algorithm. The network includes both system dynamics and cost models, used for optimal control based planning. PI-Net is fully differentiable, learning both dynamics and cost models end-to-end by back-propagation and stochastic gradient descent. Because of this, PI-Net can learn to plan. PI-Net has several advantages: it can generalize to unseen states thanks to planning, it can be applied to continuous control tasks, and it allows for a wide variety learning schemes, including imitation and reinforcement learning. Preliminary experiment results show that PI-Net, trained by imitation learning, can mimic control demonstrations for two simulated problems; a linear system and a pendulum swing-up problem. We also show that PI-Net is able to learn dynamics and cost models latent in the demonstrations.
Generally used optimal controller, linear quadratic regulator (LQR), is also differentiable and Ref. @cite_5 employs this insight to original cost models to improve short-term MPC performance. The main advantage of the path integral control over (iterative-)LQR is that we do not require a linear and quadratic approximation of non-linear dynamics and cost model. In order to differentiate iLQR with non-linear models by back-propagation, we must iteratively differentiate the functions during preceding forward pass, making the backward pass very complicated.
@cite_5
195033972 2963681938 2162919642 1991687766
This paper presents an Iterative Linear Quadratic Regulator (ILQR) method for locally-optimal feedback control of nonlinear dynamical systems. The method is applied to a musculo-skelet al arm model with 10 state dimensions and 6 controls, and is used to compute energy-optimal reaching movements. Numerical comparisons with three existing methods demonstrate that the new method converges substantially faster and finds slightly better solutions. We consider adaptive control of the Linear Quadratic Regulator (LQR), where an unknown linear system is controlled subject to quadratic costs. Leveraging recent developments in the estimation of linear systems and in robust controller synthesis, we present the first provably polynomial time algorithm that achieves sub-linear regret on this problem. We further study the interplay between regret minimization and parameter estimation by proving a lower bound on the expected regret in terms of the exploration schedule used by any algorithm. Finally, we conduct a numerical study comparing our robust adaptive algorithm to other methods from the adaptive LQR literature, and demonstrate the flexibility of our proposed method by extending it to a demand forecasting problem subject to state constraints. Aiming at the difficulty of designing weighting matrices for linear quadratic regulator (LQR), a multi-objective evolution algorithm (MOEA) based approach is proposed. The LQR weighting matrices, state feedback control rate and optimal controller are obtained by means of establishing the multi-objective optimization model of LQR weighting matrices and applying MOEA to it, which makes control system meet multiple performance indexes simultaneously. Controller of double inverted pendulum system is designed using the proposed approach. Simulation results show that it has shorter adjusting time, smaller amplitude value deviating from steady-state than the pole assignment LQR weighting matrices design approach. So the validity of the proposed approach is confirmed. In this paper the method of Linear Quadratic Regulator (LQR) is employed for controlling accelerator PWM power supply, and improving its dynamic functioning. A method based on the Particle Swarm Optimization is proposed to optimize the weighting matrix Q and R. In order to control the dynamic behavior of the power supply, the fitness function of PSO is acquired considering the needs of the power properties. The results of simulation proved that the proposed method was much better than simple LQR method and LQR-GA method, and the performance of the power supply was improved dramatically.