At gradient computation time, the actor's policy that has generated experiences might lag behind the learner's policy by updates caused by other actor's trajectories. To correct this gap, we need off-policy revisions; thus, [1]} introduced V-trace off-policy correction. Let value and policy functions parameterized by \(\theta \)  and \(\phi \) , respectively. Also, suppose a lightly older policy \(\mu \)  has stored trajectories in the replay buffer. So the n-step V-trace target for \(V(s_t;\theta )\)  is defined as:
\( v_t = V(s_t; \theta ) + \sum _{i=t}^{t+n-1} \gamma ^{i-t} \big (\prod _{j=t}^{i-1} c_j\big ) \rho _i (r_i + \gamma V(s_{i+1};\theta ) - V(s_i;\theta )), \) 
