One perspective in the literature is that Thompson sampling, and its variants, describe a way of making decisions given uncertainty and that the use of a posterior for uncertainty quantification can be swapped with some alternative. One option is to use boostrap samples rather than posterior samples [1]}, [2]}. This has been used most substantially in reinforcement learning [3]}, [4]}. Others update the parameters of a parametric posterior distribution using some alternative logic. [5]} and [6]} inflate posterior variances in order to establish worst-case guarantees for a TS-style algorithm. [7]} recently proposed a parameter updating rule motivated by doubly robust estimation techniques. An emerging literature focuses on sharp frequentist uncertainty quantification in adaptive experiments [8]}, [9]}, [10]}, [11]} and these might be combined with TS in a similar manner. It is an open question whether these techniques work effectively in the setting of this paper.
