where \(t\)  is the step counter for how many times the agent has visited \((x1, a1)\)  at step \(h=1\) , \(b_t\)  is the confidence bonus indicating the agents confidence in its \(Q\) -value at \((x1, a1, h=1)\) , and the learning rate \(\alpha \)  is \(\alpha _t := \frac{H+1}{H+t}\) .
This choice of learning rate \(\alpha _t\)  is crucial to obtain a total regret that is not exponential in \(H\)  [1]}.
