This type of methods incorporates safety constraints into the RL objective and yields a constrained sequential optimization task.
These approaches employ different optimization objectives to guide the updates of parametric policies, which can be tackled by Lagrangian relaxation [1]}, [2]} or the penalty method [3]}. Unfortunately, the “soft" loss function in the sample-based learning does not consistently enforce the “hard" constraint in practice and barely leads to zero-cost-return policies even at convergence.
