For Prioritized Experience Replay we set \(\omega =0.6\)  and linearly anneal \(\beta \)  from \(0.4\)  to 1 after \(75\%\)  of training as done in [1]}.
Experimentally these hyperparameter choices yielded the best results. Intuitively, we want to prioritize aggressively, but not dominate the sampled experiences with large TD-Error experiences.
The size of the replay buffer selected bluevaries depending on the experiment, see Section .
