Value Clipping: In addition to clipping the policy updates, our method also uses value clipping to restrict the update of critic function for each agent \(a\)  to be smaller than \(\epsilon \)  as proposed by [1]} using:
\(\mathcal {L}^a(\phi ) = \mathbb {E}_{z_t^a} \left[ \min \left\lbrace \left(V_{\phi }(z_t^a) - \hat{V}_t^a \right)^2, \left(V_{\phi _{old}}(z_t^a) + \textrm {clip}(V_{\phi }(z_t^a) - V_{\phi _{old}}(z_t^a), -\epsilon , +\epsilon ) - \hat{V}_t^a \right)^2\right\rbrace \right],\) 
