The DQN [1]} was one of the first deep RL algorithms, in which transition tuples are sampled from \(\mathcal {D}\)  to learn a function \(Q^\pi : \mathbf {S} \times \mathbf {A} \rightarrow \mathbb {R}\)  that minimizes the Bellman error by minimizing the loss function:
\(\mathcal {L}(\mathbf {s}_t, \mathbf {a}_t) =l(Q^\pi (\mathbf {s}_t, \mathbf {a}_t) -\mathbb {E}_\pi [r_t+ \gamma Q^\pi (\mathbf {s}_{t+1}, \mathbf {a}_{t+1})])\) 
