I'm trying to understand the policy gradient approach for solving the cartpole problem.In this approach, we're expressing the gradient of the loss w.r.t each parameter of our policy as an expectation of the sum of gradients of our policy gradient for all actions in a sequence, weighted by the sum of discounted rewards in that sequence:$$\nabla_\theta L(\theta) = E[ G(S_{0:T}, A_{0:T})\sum_{t=0}^{T}\nabla_\theta log\pi_\theta (A_t|S_t) ]$$and we estimate it using an empirical average across all samples in an episode - which makes sense intuitively. BUT the less intuitive part is that I saw a common practice to normalize advantages in between episodes in several implementations (and indeed it works better). So after they calculate the they wouldn't directly use the advantage, but rather would normalize it, e.g. here they do after every episode:discounted_epr = discount_rewards(epr)discounted_epr -= np.mean(discounted_epr)discounted_epr /= np.std(discounted_epr)what's the justification for that - both in theory and in intuition? It seems to me that if an episode is long and as such has large advantages, it's worth learning more from that episode than from a 3 moves episode. What am I missing? 
Why do we normalize the discounted rewards when doing policy gradient reinforcement learning?
reinforcement learning