This study lies in the context of constrained Markov Decision Process (CMDP) [1]}, which extends standard MDP [2]} as a tuple \((\mathcal {S},\mathcal {A},\mathcal {P},\mathcal {R},\mathcal {C},\mu ,\gamma )\) . \({\mathcal {S}}\)  and \({\mathcal {A}}\)  denote the state space and the action space, respectively. \({\mathcal {P}}: {\mathcal {S}} \times \mathcal {A} \times \mathcal {S} \mapsto [0,1]\)  is the transition probability function to describe the dynamics of the system. \(\mathcal {R} : \mathcal {S} \times \mathcal {A} \mapsto \mathbb {R}\)  is the reward function. \(\mathcal {C} : \mathcal {S} \times \mathcal {A} \mapsto [0,+\infty ] \)  is the cost function and reflects the safety violation. \(\mu :\mathcal {S} \mapsto [0,1]\)  is the initial state distribution, and \(\gamma \)  is the discount factor for future reward and cost.
A stationary policy \(\pi : S \mapsto P(A)\)  maps the given states to probability distributions over action space and the expected discounted return of the policy is \(J_R(\pi ) = \mathop {\mathbb {E}}_{\tau \sim \pi }\big [ \sum ^\infty _{t=0}\gamma ^t R(s_t,a_t)\big ],\)  where \(\tau \sim \pi \)  accounts for the stochastic trajectory distribution sampled on \(s_0 \sim \mu , a_t \sim \pi (\cdot | s_t), s_{t+1} \sim P(\cdot | s_t,a_t)\) . The goal of safe RL is to find the optimal policy:
\({\max }_\pi J_R(\pi )\quad \mathrm {s.t.}\ \ \pi \text{ is feasible}.\) 
