Optimal control problem involves finding controls for a dynamical system such that a certain objective function is optimized.
Traditionally, most research for solving the optimal control problems for non-linear dynamical systems uses optimal control theory which, in principle, finds optimal control by deriving Pontryagin's maximum principle or by solving the Hamilton–Jacobi–Bellman equation.
These classical strategies are offline and require a complete knowledge of system dynamics making them unsuitable for dynamical systems with uncertainties [1]}.
Recently, model predictive control (MPC) – a feedback control based on real-time optimisation – has attracted increasing attention in stochastic optimal control research [2]}, [3]}, [4]}.
Alternatively, optimal control problems could also be solved using reinforcement learning (RL) approaches. [5]} provide a benchmark study for comparison between model predictive control and model-free reinforcement learning approaches where the authors show that RL results are comparable to MPC.
Further, after a certain break-even point model-free reinforcement learning is shown to perform better than nonlinear model predictive control with an inaccurate model.
Furthermore, as opposed to MPC, RL provides an extra advantage of generality since it doesn't need complete knowledge of model dynamics.
So far, research on application of RL in optimal control problems is advancing rapidly in fields like manufacturing [6]}, energy [7]} and fluid dynamics [8]}.
However, research focused on developing RL strategy based on assessment of its robustness against the uncertainties is still an open area, especially for cases where model uncertainties has a substantial effect on the optimal control.
