Notably, the iterative executions of \(\psi (\cdot )\)  do not always converge to global (or even local) optima for the primal constrained optimization problem (REF ). Nevertheless, such a method is highly effective in practice if the initial iteration point is close to the optimal solution [1]}, [2]}. This fact emphasizes the necessity for training a pre-posed policy network via the exact penalty regularization, which provides a non-pathological initialization for USL. By means of minimizing the objective function (REF ), the output of \(\pi (\cdot |s)\)  may be still infeasible sometimes but already close to the optimal action \(a^*\) . Thus, the sequence of \(a^{k+1} = \psi (a^k)\)  is expected to converge to \(a^*\)  when \(k\rightarrow +\infty \) . However, letting \(k\rightarrow +\infty \)  is not practical in use, and thus we set an upper limit \(K\)  as the maximum iterations of USL. Note that the value of \(K\)  needs to match the gradient step-size factor \(\eta \) . For example, we set \(\eta = 0.05\)  and \(K = 20\)  to enable USL to degrade the normalized action from 1 to 0 within maximum iterations.
