In the previous section we proved the theoretical total regret of the \(Q\) -learning with Max-Optimal Initialization algorithm is bounded by \(O(\sqrt{H^2T\iota ^{\prime }})\) , while the total regret of the \(Q\) -learning with UCB-Hoeffding algorithm is bounded by \(O(\sqrt{H^4SAT\iota })\)  [1]}.
This result implies that our statement of interest holds true theoretically.
In this section our goal is to empirically show that results are consistent with our theoretical findings.
We do this by choosing a simple tabular environment to compare both algorithms.
The simple environment will allow for easier interpretation of results.
We expect the empirical results to hold for larger and more complex environments.
