Once one identifies the downstream effect of missing context as covariate shift, a natural question is whether an extension of prior covariate-shift-robust imitation learning methods to the space of histories would be able to learn effectively in partial information settings. Prior work has answered parts of this question. For example, [1]} proves that interactive imitation learning over the space of histories converges to the QMDP approximation of the expert's policy [2]}.
Recent work [3]} views these kind of partial information setting through a causal lens [4]}, and provides an algorithm (counterfactual teaching) equivalent to the interactive-expert FORWARD algorithm of [5]} under log-loss.
We build upon these analyses by providing conditions under which such approaches will converge to a policy that is equivalent in value to that of the expert in the presence of unobserved contexts.
