The nature and type of validation required of OPE estimands are clear by focusing on the assumptions, the appropriate causal estimand, and the nature of human input incorporated to address identifiability and to obtain desirable estimation properties. In particular, these desiderata further identify the nature of validation, in the form of sensitivity analysis, specifically of the type that invalidates assumptions justified for the domain, the amount and nature of unobserved confounding, or structural assumptions to clearly identify granular conditions under which the OPE estimand is meaningful. Finally, to reliably deploy policies that are evaluated in this manner, it is imperative to consider additional human factors that will allow for appropriate interpretation of recommendations inferred from the OPE estimates under the specifications outlined by our desiderata.
Note that most generally, assumptions of the data-generating process, as we have assumed throughout the draft, is a form of human expertise that allows to incorporate the most fundamental set of conditions of the domain under which the OPE estimand can be formulated. Using observational data as a means to learn such structures reliably is a nascent area called causal discovery or causal structure learning [1]}.
