A more theoretical direction for future work is to generalize our results about DTS. As one natural generalization, suppose the goal is to identify the best policy from a pre-specified class \(\Pi \) . Each element \(\pi \in \Pi \)  is a mapping from \(\mathcal {X}\)  to \([k]\) . Overloading notation, take \(\mu (\theta , \pi , w ) = \sum _{x} w(x) \mu (\theta , \pi (x), x)\)  to be the average reward if \(\pi \)  is employed throughout the population. We can naturally generalize DTS by taking \(\pi ^* = \pi ^*(\theta ) \in \operatornamewithlimits{arg\,max}_{\pi } \mu (\theta , \pi , w )\)  to be the optimal policy and \(\alpha _{t,\pi } = \mathbb {P}(\pi ^* = \pi \mid H_t)\)  to be the posterior measure of \(\pi ^*\) . So far we have studied the coarsest possible segmentation, where \(\Pi \)  consists of \(k\)  policies that prescribe a fixed arm in every context. The typical definition of TS in contextual bandits takes \(\Pi \)  to be all possible decision-rules. There is it is critical that sampling from \(\alpha _{t}\)  can be done efficiently, without actually enumerating the space of possible policies. When contexts are i.i.d, we know how to analyze DTS for any policy class by using the information theoretic analysis of [1]}, [2]}. We leave the writing of this generalization to future work. More broadly, TS has been studied in a range of complex problems, including reinforcement learning, and it would be interesting to extend the ideas in this paper to that setting.
