where \(\alpha \)  is the step size and \(Y_t\)  is called the target value. While this algorithm had been initially studied in the context of a tabular representation of Q for discrete states and actions, in many practical applications the Q value is approximated by a learned function. Since the emergence of deep learning, the preferred approximation technique is based on a deep neural network. DQN [1]}, the first deep RL algorithm based on these ideas had demonstrated super-human performance in Atari Games, but required a very large number of training iterations. From this baseline, subsequent algorithms improved both the learning speed and achievable performance, with one of the main means for this being techniques to reduce the overestimation bias of the Q-function.
