Our focus on the contextual MDP [1]} is because, as argued by [2]}, [3]}, context that is updated throughout the episode and is only reflected for a single step prevents the learner from refining its estimates via considering history.
Our results apply to settings beyond the CMDP where a similar asymptotic realizability property holds. [4]} consider IL in contextual MDPs but give the learner access to the confounder at test time and do not consider policies that operate over the space of histories. [5]} also consider imitation learning from a causal inference perspective but focus on issues of covariate shift [6]}, rather than those of missing information.
