The scheme of the proposed method is related to the previous research
in the guided policy search methods, but there are technical differences.
The works of [1]}, [2]} utilized the DDP method and [3]}
worked with Newton's method.
On the other hand, we employ a convex approximation of the original
nonconvex problem instead of solving the nonconvex optimal control problem in each iteration.
There are two advantages of this approach. First, solving the convex
approximation instead of solving the nonconvex trajectory optimization problem is more computationally tractable. Since this trajectory optimization
is repeated as the first step in each iteration, solving the nonconvex
problem is too costly. More importantly, leveraging convex optimization
makes it straightforward to impose state and input constraints while
most earlier works did not impose the constraints. For instance, the work in [4]} imposed the state constraints by using an augmented Lagrangian formulation, but the constraint exists for the training phase, and the final policy does not activate the constraint. The work in the imitation learning [2]} considered both inequality and equality constraints, but the inequality constraint are implemented as a penalty via a barrier function, not as a hard constraint.
