Recall that our algorithm makes two important changes to the \(Q\) -learning with UCB-Hoeffding algorithm from [1]}. Namely, the initialization is changed according to equation (REF )
and the \(Q\) -function is only updated if the current state, action, and step is \((x1, a1, h=1)\) .
We refer to these two changes as assumption 1 (A1) and assumption 2 (A2) respectively.
We train an agent using three different algorithms:
