[Notes on the proof]
First, suppose that there are no contexts. That is, assume \(X_t\in \mathbb {R}\)  and \(X_t=1\)  for each \(t\)  almost surely. In that case DTS is just the top-two Thompson sampling algorithm proposed in [1]}. Argument like the ones in [1]} imply almost sure convergence to the optimal proportions, i.e. \(\lim _{t\rightarrow \infty } p_{t,i}=p^*_i(\theta _0)\)  with probability 1. This means there exists \(T\)  with \(\mathbb {P}(T<\infty \mid \theta =\theta _0)=1\)  such that (REF ) holds for \(t\geqslant T\) . [3]} studies a different top-two sampling algorithm and shows one can ensure \(\mathbb {E}[T]<\infty \) . [4]} show this kind of result for top-two Thompson sampling.
