A Survey on Causal Inference <s> Meta-Learning Methods <s> One type of semiparametric regression is b8X A u(Z), where b and u(Z) are an unknown slope coefficient vector and function. Estimates of b based on incorrect parametrization of u are generally inconsist ent, whereas consistent nonparametric estimates converge slowly. An e stimate, bC, is constructed by inserting nonpar-ametric regression es timates in the nonlinear orthogonal projection on Z. Under regularity conditions bC is shown to be N1/2-consistent for b and asymptoticall y normal, and a consistent estimate of its limiting covariance matrix is given. The author discusses the identification problem and bC's e fficiency. Extensions to other econometric models are described. Copyright 1988 by The Econometric Society. <s> BIB001 </s> A Survey on Causal Inference <s> Meta-Learning Methods <s> This paper presents an implementation of matching estimators for average treatment effects in Stata. The nnmatch command allows you to estimate the average effect for all units or only for the treated or control units; to choose the number of matches; to specify the distance metric; to select a bias adjustment; and to use heteroskedastic-robust variance estimators. Copyright 2004 by StataCorp LP. <s> BIB002 </s> A Survey on Causal Inference <s> Meta-Learning Methods <s> We develop a Bayesian "sum-of-trees" model, named BART, where each tree is constrained by a prior to be a weak learner. Fitting and inference are accomplished via an iterative backfitting MCMC algorithm. This model is motivated by ensemble methods in general, and boosting algorithms in particular. Like boosting, each weak learner (i.e., each weak tree) contributes a small amount to the overall model. However, our procedure is defined by a statistical model: a prior and a likelihood, while boosting is defined by an algorithm. This model-based approach enables a full and accurate assessment of uncertainty in model predictions, while remaining highly competitive in terms of predictive accuracy. <s> BIB003 </s> A Survey on Causal Inference <s> Meta-Learning Methods <s> Flexible estimation of heterogeneous treatment effects lies at the heart of many statistical challenges, such as personalized medicine and optimal resource allocation. In this paper, we develop a general class of two-step algorithms for heterogeneous treatment effect estimation in observational studies. We first estimate marginal effects and treatment propensities in order to form an objective function that isolates the causal component of the signal. Then, we optimize this data-adaptive objective function. Our approach has several advantages over existing methods. From a practical perspective, our method is flexible and easy to use: In both steps, we can use any loss-minimization method, e.g., penalized regression, deep neutral networks, or boosting; moreover, these methods can be fine-tuned by cross validation. Meanwhile, in the case of penalized kernel regression, we show that our method has a quasi-oracle property: Even if the pilot estimates for marginal effects and treatment propensities are not particularly accurate, we achieve the same error bounds as an oracle who has a priori knowledge of these two nuisance components. We implement variants of our approach based on both penalized regression and boosting in a variety of simulation setups, and find promising performance relative to existing baselines. <s> BIB004 </s> A Survey on Causal Inference <s> Meta-Learning Methods <s> There is growing interest in estimating and analyzing heterogeneous treatment effects in experimental and observational studies. We describe a number of metaalgorithms that can take advantage of any supervised learning or regression method in machine learning and statistics to estimate the conditional average treatment effect (CATE) function. Metaalgorithms build on base algorithms—such as random forests (RFs), Bayesian additive regression trees (BARTs), or neural networks—to estimate the CATE, a function that the base algorithms are not designed to estimate directly. We introduce a metaalgorithm, the X-learner, that is provably efficient when the number of units in one treatment group is much larger than in the other and can exploit structural properties of the CATE function. For example, if the CATE function is linear and the response functions in treatment and control are Lipschitz-continuous, the X-learner can still achieve the parametric rate under regularity conditions. We then introduce versions of the X-learner that use RF and BART as base learners. In extensive simulation studies, the X-learner performs favorably, although none of the metalearners is uniformly the best. In two persuasion field experiments from political science, we demonstrate how our X-learner can be used to target treatment regimes and to shed light on underlying mechanisms. A software package is provided that implements our methods. <s> BIB005
When designing the heterogeneous treatment effect estimation algorithms, two key factors should be considered: 1) Control the confounders, i.e., eliminate the spurious correlation between the confounder and the outcome; 2) Give an accurate expression of the CATE estimation BIB004 . The methods mentioned in the previous sections seek to satisfy the two requirements simultaneously, while meta-learning based algorithms separate them into two steps. In general, the meta-learning based algorithms have the following procedures: (1) Estimate the conditional mean outcome E[Y |X = x], and the prediction model learned in this step is the base learner. (2) Derive the CATE estimator based on the difference of results obtained from step BIB002 . Existing meta-learning methods include T-learner BIB005 , S-learner BIB005 , X-learner BIB005 , U-learner BIB004 and R-learner BIB004 , which are introduced in the following. In detail, the T-learner BIB005 adopts two trees to estimate the conditional treated/control outcomes, which are denoted as µ 0 ( , respectively. Letμ 0 (x) andμ 0 (x) denote the trained tree model on the control/treated group. Then the CATE of T-learner estimation is obtained as: τ T (x) =μ 1 (x) −μ 0 (x). T-learner trains two base models for control and treated groups (the name "T" comes from two base model), while S-learner BIB005 views the treatment assignment as one feature and estimate the combined outcome as: µ(x, w) = E[Y F |X = x,W = w] (The name "S" denotes single). µ(x, w) can be any base model, and we denote the trained model asμ(x, w). The CATE estimator provided by S-learner is then given as: However, T-learner and S-learner highly rely on the performance of the trained base models. When the number of units in two groups are extremely unbalanced (i.e., the number of one group is much larger than the other), the performance of the base model trained on the small group would be poor. To overcome this problem, X-learner BIB005 is proposed, which adopts information from the control group to give a better estimator on the treated group and vice versa. The cross-group information usage is where X-learner comes from, and the X denotes "cross group". In detail, X-learner contains three key steps. The first step of X-learner is the same as T-learner, and the trained base learners are denoted asμ 0 (x) andμ 1 (x). In the second step, X-learner calculates the difference between the observed outcome and the estimated outcome as the imputed treatment effect: In the control group, the difference is the estimated treated outcome subtracts the observed control outcome, denoted as: Similarly, in the treated group, the difference is formulated as: After the difference calculation, the dataset is transformed into two groups with imputed treatment effect: control group: (X C ,D C ) and treated group: (X T ,D T ). On two imputed datasets, the two base learners of treatment effect τ 1 (x)( τ 0 (x)) are trained with X C (X T ) as the input andD C (D T ) as the output. The last step is to combine the two CATE estimators by weighted average: τ X (x) = д(x)τ 0 (x) + (1 − д(x))τ 1 (x), where д(x) is the weighting function ranging from 0 to 1. Overall, with the cross information usage and the weighted combination of two CATE base estimator, X-learners can handle the case where the number of units in two groups are unbalanced BIB005 . Different from the regular loss function adopted in X-learner, R-learner, proposed in BIB004 designs the loss function for CATE estimator based on the Robinson transformation BIB001 . The character "R" in R-learner denotes the Robinson transformation. The Robinson transformation can be derived by rewriting the observed outcome and the conditional outcome: Rewrite the observed outcome as: whereμ 0 is the already-trained control outcome estimator(base learner), τ (x i ) is the CATE estimator, and E[ϵ i (w i )|x i , w i ] = 0 (under ignorability). The conditional mean outcome can be also rewritten as: whereê(x) is the already-trained propensity score estimator(base learner). Robinson transformation is obtained by subtracting Eqn. BIB003 and Eqn. (29): Based on the Robinson transformation, a good CATE estimator should minimize the difference between Y F i −m(x i ) and (w i −ê(x i ))τ (x i ). Therefore, the objective function of R-learner is as follows: wherem(x i ) andê(x i ) are pre-trained outcome estimator and propensity score estimator, and Λ(τ (·)) is the regularization on τ (·).