
since \(n_{jh}^{k_{j + 1}}(s, a) \le N_{k_j h}^{\prime }(s, a)\) , due to the doubling scheme. The penultimate inequality follows from Lemma 19 of [1]}, which is restated in Appendix REF  (Lemma REF ).
