According to the potential outcomes framework [1]}, [2]}, [3]}, we define \(Y_{t,i}(A_{t,i})\)  to be the random reward variable representing the outcome that would be observed if participant \(i\)  at step \(t\)  were assigned to arm \(A_{t,i}\) . Note that each participant can be assigned only one arm, thus, we observe only one realized outcome for that participant. Denoted by \(\mathcal {H}_{t-1} \doteq \lbrace A_{\tau ,i}, Y_{\tau ,i}(A_{\tau ,i}), i=1,\dots , n, \tau = 1,\dots ,t-1 \rbrace \) , the history of selected arms and associated rewards prior to current step \(t\) , the sampling probabilities are a function of this history, i.e., \(\pi _{t,k} \doteq \mathbb {P}(A_{t,i}=k | \mathcal {H}_{t-1})\) , for \(k = 0,\dots , K\)  and \(t=0,\dots ,T\) . We assume that the reward variables are drawn independently from an unknown but fixed distribution with the following conditional mean:
\(\mathbb {E}(Y_{t,i}(A_{t,i})| \mathcal {H}_{t-1}) = \mathbb {E}(Y_{t,i}| \mathcal {H}_{t-1}, A_{t,i}) = \sum _{k=0}^{K}\mu _{t,k}\mathbb {I}(A_{t,i}=k),\) 
