and the standard \(do()\) -calculus rules [1]}. Intuitively, we are leveraging the fact that the learner's actions have all their dependence on the context mediated through the history of states to ignore them in our posterior calculation. Notice that this expression matches the off-policy posterior except for the term in green: because the on-policy learner knows that the actions in the history were not produced by the expert, it does not weight them by the expert's probability of playing them in its posterior update. Graphically, as far as the posterior over the context is concerned, we could be in the following SCM.
<FIGURE>