Policy gradient [1]} is one of the most important approaches to DRL that synthesizes policies for continuous decision making problems. For control tasks, policy gradient method and its variants have successfully synthesized neural network controllers to accomplish complex control goals [2]} without solving potentially non-linear planing problems at test time [3]}.
However, most of these methods focus on maximizing the reward function which only indirectly enforce desirable properties. Specifically, global stability of the closed-loop system [4]} guarantees convergence to the desired state of origin from any initial state and therefore is a very important property for safety critical systems (e.g. aircraft control [5]}) where not a single diverging trajectory is acceptable.
However, the set of parameters corresponding to stabilizing controllers is in general nonconvex even in the simple setting of linear systems with linear controllers [6]}, which poses significant computational challenges for neural network controllers under the general setting of nonlinear systems.
