Table REF  presents a selection of algorithms that fit into our framework and their regret bounds under delayed feedback using both active and lazy updating. Here, we see that acting in delayed environments causes an additive increase in regret for all optimistic algorithms considered. For active updating, this additive increase scales with: \(\max \lbrace C, B\rbrace HSA \mathbb {E}[\tau ]\) , where \(C\)  and \(B\)  are quantities depending on the base algorithm of choice. Typically, \(C\)  and \(B\)  involve the horizon, \(H\) , and the number of states, \(S\) . For lazy updating, this additive increase scales with: \(H^2 SA \log (K/SA)\mathbb {E}[\tau ]\) , regardless of the chosen base algorithm. In either case, the results mirror what is seen in the bandit setting, most algorithms incur an additive regret penalty of \(A\mathbb {E}[\tau ]\)  [1]}.
