We use \(\pi _w(\cdot )\)  to denote that the policy is parameterized by \(w \in \mathbb {R}^d\)  (i.e., the weights and biases of the neural network in deep RL). The agent is trained with an objective function defined as [1]}
\(J(w)~ \dot{=} ~ V^{\pi _w}(s_0), \) 
