across all rounds \(t \ge 0\) .


Proving the lemma requires several adjustments to existing arguments for deriving confidence sets in the immediate feedback setting. Firstly, we account for regularising the log-likelihood function, which we do in Lemmas REF  and REF  in Appendix . These lemmas allow us to separate noise-related terms from those introduced by biasing our estimator with the regularisation term. Subsequently, we show that the noise-related terms satisfy the martingale property under the information structure created by the delays. This result allows us to apply existing results for self-normalising processes [1]}, [2]}. See Appendix  for a full proof.
