To reach the optimal policy, estimating the action-value function \(Q_{\pi }\)  of the optimal policy is sufficient, \(Q_{*}(s, s) = \underset{\pi }{\mathop {\mathrm {argmax}}} \ Q_{\pi }(s, a)\) . We could use the Bellman equation [1]}, which is a recursive formulation relating our current value to the value we make in the immediate future:
\( Q_{*} (s, a) = \mathbb {E}[r(s, a)] + \gamma \underset{a^{\prime }}{\max }\ \mathbb {E}[Q_{*} (s^{\prime }, a^{\prime })], \) 
