Prioritized Experience Replay (PER).
During the training (cf Section ) the agent interacts with the environment by observing state \(s_t\)  and performing action \(a_t\) , either randomly with probability \(\varepsilon \)  or greedily with probability \((1-\varepsilon )\)  by selecting the action with the highest \(Q\)  value, then the environment transitions to state \(s_{t+1}\)  and the agent receives reward \({blue}{r}_{t+1}\) . Thus, the agent experiences a sequence of transition tuples \((s_t, a_t, r_{t+1}, s_{t+1})\)  which are stored in a replay buffer \(\mathcal {B}\) . The loss function of Eq. (REF ) is obtained as the expectation over a batch of such tuples capturing the experiences of the agent. Every time step \(t\)  a batch of experiences is sampled from blue\(\mathcal {B}\)  and used to update the Double DQN parameters using Prioritized Experience Replay (PER) [1]}.
