where \(Q(s_t, a_t)\)  is the expected reward of taking action \(a_t\)  at state \(s_t\)  at time step \(t\) ,
\(r_{t+1}\)  is the reward received at the next time step after taking action \(a_t\) , \(\gamma \)  is the discount factor that trades off the importance of immediate and later rewards,
\(r_{t+1} + \gamma {max}_{a^{^{\prime }}}Q(s_{t+1}, a^{^{\prime }})\)  is the TDTemporal-Difference (TD) learning refers to algorithms where estimates at the current time step are based on those of the next time step [1]}.-Target,
\(r_{t+1} + \gamma {max}_{a^{^{\prime }}}Q(s_{t+1}, a^{^{\prime }}) - Q(s_t, a_t)\)  blueis the TD-Error blue\(\delta \) , and \(0 < \alpha \le 1\)  is a constant that determines how fast the agent forgets past experiences.
Contextually, given state \(s_t\) , Eq. (REF ) improves the estimate of the value associated with action \(a_t\) . Note Eq. (REF ) does not use any probabilities defined by the MDP.
