where \(\beta \)  is the learning rate of the optimization algorithm. The gradient of an objective function can be computed using the policy gradient theorem [1]} as follows
\(\nabla _w V^{\pi _w}(s_0) = \mathbb {E}_{\pi _w}\big [ \nabla _w \big (\log ~\pi _w(s,a) \big ) Q^{\pi _w}(s,a)]. \) 
