Somewhat orthogonal to our paper, but also featuring words like “confounding”, there is a substantial and growing literature on policy evaluation from logged contextual bandit data [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}, [8]}, [9]}, [10]}, [11]}. The work of [12]} studies a setting where logged data, which might be subject to unobserved confounding, is used to warm-start Thompson sampling. Unlike this line of work, we consider the design of an adaptive experimentation algorithm, so the way decisions are made is changing but under our control.
