Motivated by the shortcoming of imitation learning, interactive
optimization approaches, also known as guided policy search, have recently been studied in [1]}, [2]}.
These methods formulate the entire policy optimization problem as two separated optimizations: one is to optimize the multiple trajectories from different initial conditions, and the other is to train the neural net policy using supervised learning with the sample trajectories generated in the previous step. Unlike imitation learning, the trajectory optimization in the first step has an additional constraint that enforces the updated trajectory to be close enough to the trajectory from the policy. Imposing this constraint can help the single neural network policy to reproduce all trajectories with the long-time horizon performance.
