Using Lemma REF  and the arguments in its proof ensures that the number of epochs is logarithmic in the number of episodes and that we can make use of Lemma 19 of [1]} to bound the sum of the bonuses. Therefore, we can bound the regret of model-based algorithms using lazy updating to handle the delayed feedback.
