Conventionally, OPE focuses on estimating the population and sub-population-level prospective performance of an evaluation policy based on data collected from a different behavior policy [1]}, [2]}. Evaluating prospective performance on (sub)-populations is akin to answering the question “Will new patients have better outcomes if treated with A instead of B?”. Yet in practice, we are also interested in questions of the form “If we had acted differently, would patient X's condition have improved?”.
The two key distinctions are that these are inherently require reasoning about retrospective performance of an evaluation policy as well as require reasoning on an individual level [3]}, [4]}, [5]}. For prospective reasoning specifically at population and sub-population levels, there is a vast literature in identifying population and sub-population off-policy estimates, including characterizing their statistical properties [6]}, [7]}, [8]}, [9]}, [10]}.
Formalizing the latter (i.e. individual level, retrospective analysis) requires counterfactual reasoning. We argue that there is a need to broaden the traditional purview of OPE to include and enable individual-level reasoning, and formally, counterfactual analysis. We posit that i) this perspective is critical to understanding the limits of current ML-based modeling approaches for OPE, and ii) allows to generalize OPE to answer novel objectives of practical interest.
