A Survey on Causal Inference <s> Distance Metric. <s> Several matching methods that match all of one sample from another larger sample on a continuous matching variable are compared with respect to their ability to remove the bias of the matching variable. One method is a simple mean-matching method and three are nearest available pair-matching methods. The methods' abilities to remove bias are also compared with the theoretical maximum given fixed distributions and fixed sample sizes. A summary of advice to an investigator is included. <s> BIB001 </s> A Survey on Causal Inference <s> Distance Metric. <s> Abstract The propensity score is the conditional probability of assignment to a particular treatment given a vector of observed covariates. Previous theoretical arguments have shown that subclassification on the propensity score will balance all observed covariates. Subclassification on an estimated propensity score is illustrated, using observational data on treatments for coronary artery disease. Five subclasses defined by the estimated propensity score are constructed that balance 74 covariates, and thereby provide estimates of treatment effects using direct adjustment. These subclasses are applied within sub-populations, and model-based adjustments are then used to provide estimates of treatment effects within these sub-populations. Two appendixes address theoretical issues related to the application: the effectiveness of subclassification on the propensity score in removing bias, and balancing properties of propensity scores with incomplete data. <s> BIB002 </s> A Survey on Causal Inference <s> Distance Metric. <s> Abstract A comparison and evaluation is made of recent proposals for multivariate matched sampling in observational studies, where the following three questions are answered: (1) Algorithms: In current statistical practice, matched samples are formed using “nearest available” matching, a greedy algorithm. Greedy matching does not minimize the total distance within matched pairs, though good algorithms exist for optimal matching that do minimize the total distance. How much better is optimal matching than greedy matching? We find that optimal matching is sometimes noticeably better than greedy matching in the sense of producing closely matched pairs, sometimes only marginally better, but it is no better than greedy matching in the sense of producing balanced matched samples. (2) Structures: In common practice, treated units are matched to one control, called pair matching or 1–1 matching, or treated units are matched to two controls, called 1–2 matching, and so on. It is known, however, that the optimal st... <s> BIB003 </s> A Survey on Causal Inference <s> Distance Metric. <s> Matched sampling is a standard technique in the evaluation of treatments in observational studies. Matching on estimated propensity scores comprises an important class of procedures when there are numerous matching variables. Recent theoretical work (Rubin, D. B. and Thomas, N., 1992, The Annals of Statistics 20, 1079-1093) on affinely invariant matching methods with ellipsoidal distributions provides a general framework for evaluationg the operating characteristics of such methods. Moreover, Rubin and Thomas (1992, Biometrika 79, 797-809) uses this framework to derive several analytic approximations under normality for the distribution of the first two moments of the matching variables in samples obtained by matching on estimated linear propensity scores. Here we provide a bridge between these theoretical approximations and actual practice. First, we complete and refine the nomal-based analytic approximations, thereby making it possible to apply these results to practice. Second, we perform Monte Carlo evaluations of the analytic results under normal and nonnormal ellipsoidal distributions, which confirm the accuracy of the analytic approximations, and demonstrate the predictable ways in which the approximations deviate from simulation results when normal assumptions are violated within the ellipsoidal family. Third, we apply the analytic approximations to real data with clearly nonellipsoidal distributions, and show that the thoretical expressions, although derived under artificial distributional conditions, produce useful guidance for practice. Our results delineate the wide range of settings in which matching on estimated linear propensity scores performs well, thereby providing useful information for the design of matching studies. When matching with a particular data set, our theoretical approximations provide benchmarks for expected performance under favorable conditions, thereby identifying matching variables requiring special treatment. After matching is complete and data analysis is at hand, our results provide the variances required to compute valid standard errors for common estimators. <s> BIB004 </s> A Survey on Causal Inference <s> Distance Metric. <s> Abstract Propensity score matching refers to a class of multivariate methods used in comparative studies to construct treated and matched control samples that have similar distributions on many covariates. This matching is the observational study analog of randomization in ideal experiments, but is far less complete as it can only balance the distribution of observed covariates, whereas randomization balances the distribution of all covariates, both observed and unobserved. An important feature of propensity score matching is that it can be easily combined with model-based regression adjustments or with matching on a subset of special prognostic covariates or combinations of prognostic covariates that have been identified as being especially predictive of the outcome variables. We extend earlier results by developing approximations for the distributions of covariates in matched samples created with linear propensity score methods for the practically important situation where matching uses both the estimat... <s> BIB005 </s> A Survey on Causal Inference <s> Distance Metric. <s> This paper considers different methods for solving the evaluation problem. I highlight the role of heterogeneity in program impacts in defining evaluation parameters of interest and in interpreting estimated program impacts. I discuss the strengths and weaknesses of social experiments and conclude that they require careful implementation and interpretation. I review and critique two popular non-experimental evaluation methods: difference-in-differences and propensity score matching. I find that the former relies on assumptions at odds with the empirical data and that the latter is not a magical solution to all evaluation problems. Finally, I argue for the importance of paying attention to data quality and general equilibrium effects. <s> BIB006 </s> A Survey on Causal Inference <s> Distance Metric. <s> Randomized experiments are desirable but not always possible for social policy research, so "quasi-experiments" have become popular alternatives. This article reports on a systematic review of nearly three decades of methodological research to address the question of whether there is evidence justifying quasi-experiments for estimating the impacts of education, training, and welfare programs on earnings. <s> BIB007 </s> A Survey on Causal Inference <s> Distance Metric. <s> Propensity Score Matching (PSM) has become a popular approach to estimate causal treatment effects. It is widely applied when evaluating labour market policies, but empirical examples can be found in very diverse fields of study. Once the researcher has decided to use PSM, he is confronted with a lot of questions regarding its implementation. To begin with, a first decision has to be made concerning the estimation of the propensity score. Following that one has to decide which matching algorithm to choose and determine the region of common support. Subsequently, the matching quality has to be assessed and treatment effects and their standard errors have to be estimated. Furthermore, questions like "what to do if there is choice-based sampling?" or "when to measure effects?" can be important in empirical studies. Finally, one might also want to test the sensitivity of estimated treatment effects with respect to unobserved heterogeneity or failure of the common support condition. Each implementation step involves a lot of decisions and different approaches can be thought of. The aim of this paper is to discuss these implementation issues and give some guidance to researchers who want to use PSM for evaluation purposes. <s> BIB008 </s> A Survey on Causal Inference <s> Distance Metric. <s> The propensity score collapses the covariates of an observational study into a single measure summarizing their joint association with treatment conditions; prognostic scores summarize covariates' association with potential responses. As with propensity scores, stratification on prognostic scores brings to uncontrolled studies a concrete and desirable form of balance, a balance that is more familiar as an objective of experimental control. Like propensity scores, prognostic scores can reduce the dimension of the covariate, yet causal inferences conditional on them are as valid as are inferences conditional only on the unreduced covariate. As a method of adjustment unto itself, prognostic scoring has limitations not shared with propensity scoring, but it holds promise as a complement to the propensity score, particularly in certain designs for which unassisted propensity adjustment is difficult or infeasible. Copyright 2008, Oxford University Press. <s> BIB009 </s> A Survey on Causal Inference <s> Distance Metric. <s> We discuss a method for improving causal inferences called ‘‘Coarsened Exact Matching’’ (CEM), and the new ‘‘Monotonic Imbalance Bounding’’ (MIB) class of matching methods from which CEM is derived. We summarize what is known about CEM and MIB, derive and illustrate several new desirable statistical properties of CEM, and then propose a variety of useful extensions. We show that CEM possesses a wide range of statistical properties not available in most other matching methods but is at the same time exceptionally easy to comprehend and use. We focus on the connection between theoretical properties and practical applications. We also make available easy-to-use open source software for R, Stata, and SPSS that implement all our suggestions. <s> BIB010 </s> A Survey on Causal Inference <s> Distance Metric. <s> When estimating causal effects using observational data, it is desirable to replicate a randomized experiment as closely as possible by obtaining treated and control groups with similar covariate distributions. This goal can often be achieved by choosing well-matched samples of the original treated and control groups, thereby reducing bias due to the covariates. Since the 1970’s, work on matching methods has examined how to best choose treated and control subjects for comparison. Matching methods are gaining popularity in fields such as economics, epidemiology, medicine, and political science. However, until now the literature and related advice has been scattered across disciplines. Researchers who are interested in using matching methods–or developing methods related to matching–do not have a single place to turn to learn about past and current research. This paper provides a structure for thinking about matching methods and guidance on their use, coalescing the existing research (both old and new) and providing a summary of where the literature on matching methods is now and where it should be headed. <s> BIB011 </s> A Survey on Causal Inference <s> Distance Metric. <s> A widely used method for estimating counterfactuals and causal treatment effects from observational data is nearest-neighbor matching. This typically involves pairing each treated unit with its nearest-in-covariates control unit, and then estimating an average treatment effect from the set of matched pairs. Although straightforward to implement, this estimator is known to suffer from a bias that increases with the dimensionality of the covariate space, which can be undesirable in applications that involve high-dimensional data. To address this problem, we propose a novel estimator that first projects the data to a number of random linear subspaces, and it then estimates the median treatment effect by nearest-neighbor matching in each subspace. We empirically compute the mean square error of the proposed estimator using semi-synthetic data, and we demonstrate the method on real-world digital marketing campaign data. The results show marked improvement over baseline methods. <s> BIB012 </s> A Survey on Causal Inference <s> Distance Metric. <s> I show that for a linear model and estimating a coefficient on an endogenous explanatory variable, adding covariates that satisfy instrumental variables assumptions increases the amount of inconsistency. A special case is an endogenous binary treatment and estimating a constant treatment effect when matching on covariates that satisfy instrumental variables, rather than ignoribility, assumptions. I also establish a general result that implies that regression adjustment using the propensity score based on instrumental variables actually maximizes the inconsistency among regression-type estimators. <s> BIB013 </s> A Survey on Causal Inference <s> Distance Metric. <s> Estimating treatment effects from observational data is challenging due to the missing counterfactuals. Matching is an effective strategy to tackle this problem. The widely used matching estimators such as nearest neighbor matching (NNM) pair the treated units with the most similar control units in terms of covariates, and then estimate treatment effects accordingly. However, the existing matching estimators have poor performance when the distributions of control and treatment groups are unbalanced. Moreover, theoretical analysis suggests that the bias of causal effect estimation would increase with the dimension of covariates. In this paper, we aim to address these problems by learning low-dimensional balanced and nonlinear representations (BNR) for observational data. In particular, we convert counterfactual prediction as a classification problem, develop a kernel learning model with domain adaptation constraint, and design a novel matching estimator. The dimension of covariates will be significantly reduced after projecting data to a low-dimensional subspace. Experiments on several synthetic and real-world datasets demonstrate the effectiveness of our approach. <s> BIB014 </s> A Survey on Causal Inference <s> Distance Metric. <s> Estimating the individual treatment effect (ITE) from observational data is essential in medicine. A central challenge in estimating the ITE is handling confounders, which are factors that affect both an intervention and its outcome. Most previous work relies on the unconfoundedness assumption, which posits that all the confounders are measured in the observational data. However, if there are unmeasurable (latent) confounders, then confounding bias is introduced. Fortunately, noisy proxies for the latent confounders are often available and can be used to make an unbiased estimate of the ITE. In this paper, we develop a novel adversarial learning framework to make unbiased estimates of the ITE using noisy proxies. <s> BIB015
Various distances have been adopted to compare the closeness between units BIB003 , such as the widely used Euclidean distance BIB001 and Mahalanobis distance BIB005 . Meanwhile, many matching methods develop their own distance metrics, which can be abstracted as: The existing distance metrics mainly vary in how to design the transformation function f (·). Propensity score based transformation. Original covariates of units can be represented by propensity scores. As a result, the similarity between two units can be directly calculated as: where e i , and e j are the propensity scores of x i and x j , respectively. Later, the linear propensity score based distance metric is also proposed, which is defined as D(x i , x j ) = |logit(e i ) − logit(e j )|. This improved version is recommended since it can effectively reduce the bias BIB011 . Furthermore, the propensity score based distance metric can be combined with other existing distance metrics, which provides a fine-grained comparison. In BIB005 , when the difference of two unit's propensity scores is within a certain range, they are further compared with other distances on some key covariates. Under this metric, the closeness of two units contains two criteria: they are relatively close under propensity score measure, and they particularly similar under the comparison of the key covariates BIB011 . Other transformations. Propensity score only adopts the covariate information, while some other distance metrics are learned by utilizing both the covariates and the outcome information so that the transformed space can preserve more information. One representative metric is the prognosis score BIB009 , which is the estimated control outcome. The transformation function is represented as: f (x) =Ŷ c . However, the performance of the prognosis score relies on modeling the relationship between the covariates and control outcomes. Moreover, the prognosis score only takes the control outcome into consideration and ignores the treated outcome. The Hilbert-Schmidt Independence Criterion based nearest neighbor matching (HSIC-NNM) proposed in  could overcome the drawbacks of prognosis score. HSIC-NNM learns two linear projections for control outcome estimation task and treated outcome estimation task separately. To fully explore the observed control/treated outcome information, the parameters of linear projection is learned by maximizing the nonlinear dependency between the projected subspace and the outcome: , where w = 0, 1 represent the control group and treated group, respectively. X w M w is the transformed subspace with the transformation function as: w is the observed control/treated outcome, and R is the regularization to avoid overfitting. The objective function ensures the learned transformation functions project the original covariates to an information subspace where similar units will have similar outcomes. Compared with propensity score based distance metric that focuses on balancing, prognosis score and HSIC-NNM focus on embedding the relationship between the transformed space and the observed outcome. These two lines of methods have different advantages, and some recent work tries to integrate these advantages together. In BIB014 , the balanced and nonlinear representation (BNR) is proposed to project the covariates into a balanced low-dimensional space. In detail, the parameters in the nonlinear transformation function is learned by jointly optimizing the following two objectives: (1) Maximizing the differences of noncontiguous-class scatter and within-class scatter so that the units with the same outcome prediction shall have similar representations after transformation; (2) Minimizing the maximum mean discrepancy between the transformed control and outcome group in order to get the balanced space after transformation. A series of works that have similar objectives but vary in balancing regularization have been proposed, such as using the conditional generative adversarial network to ensure the transformation function blocks the treatment assignment information BIB015  . The methods mentioned above adopt either one or two transformations for treated and control groups separately. Different from the existing method, Randomized Nearest Neighbor Matching (RNNM) BIB012 adopts a number of random linear projections as the transformation function, and the treatment effects are obtained as the median treatment effect by nearest-neighbor matching in each transformed subspace. The theoretical motivation of this approach is the Johnson-Lindenstrauss (JL) lemma, which guarantees that the pairwise similarity information of the points in the high-dimensional space can be preserved through random linear projection. Powered by the JL lemma, RNNM ensembles the treatment effect estimation results of several linear random transformations. 3.3.2 Choosing a Matching Algorithm. After defining the similarity metric, the next step is to find the neighbors. In BIB008 , existing matching algorithms are divided into four essential approaches, including the nearest neighbor matching, caliper, stratification and kernel, as shown in Fig. 3 . The most straightforward matching estimator is nearest neighbour matching (NNM). In particular, a unit in the control group is chosen as the matching partner for a treated unit, so that they are closest based on a similarity score (e.g., propensity score). The NNM has several variants like NNM with replacement and NNM without replacement. Treated units are matched to one control, called pair matching or 1-1 matching, or treated units are matched to two controls, called 1-2 matching, and so on. It's a trade-off to determine the number of neighbors, since a large number of neighbors may result in the treatment effect estimator with high bias but low variance, while small number results in low bias but high variance. It is known, however, that the optimal structure is a full matching in which a treated unit may have one or several controls or a control may have one or several treated units BIB003 . NNM may have bad matches if the closest partner is far away. One can set a tolerance level on the maximum propensity score distance (caliper) to avoid this problem. Hence, caliper matching is one form of imposing a common support condition. The stratification matching is to partition the common support of the propensity score into a set of intervals and then to take the mean difference in outcomes between treated and control observations in order to calculate the impact within each interval. This method is also known as interval matching, blocking and subclassification  . The matching algorithms discussed above have in common that only a few observations in the control group are used to create the counterfactual outcome of a treatment observation. Kernel matching (KM) and local linear matching (LLM) are nonparametric matching that use weighted averages of observations in the control group to create the counterfactual outcome. Thus, one major advantage of these approaches is the lower variance, because we use more information to create counterfactual outcome. Here, we also want to introduce another matching method called Coarsened Exact Matching (CEM) proposed in BIB010 . Because either the 1-k matching or the full matching fails to consider the extrapolation region, where few or no reasonable matches exist in the other treatment group, CEM was proposed to handle this problem. CEM first coarsen the selected important covariate,i.e., discretization, and then perform exact matching on the coarsened covariates. For example, if the selected covariates are age (age > 50 is 1, and others are 0) and gender (female as 1, and male as 0). A female patient with age 50 in the treated group is represented by the coarsen covariates as (1, 1). She will only match the patients in the treated group with exactly the same coarsened covariates value. After exact matching, the whole data is separated into two subsets. In one subset, every unit has its exact matched neighbors and it is the opposite in the other subset which contains the units in the extrapolation region. The outcomes of units in the extrapolation region are estimated by the outcome prediction model trained on the matched subset. So far, the treatment effect on the two subsets can be estimated separately, and the final step is to combine treatment effect on the two subsets by weighted average. We have provided several different matching algorithms, but the most important question is how we should select a perfect matching method. Asymptotically all matching methods should yield the same results as the sample size grows and they will become closer to comparing only exact matches BIB006 . When we only have small samples size, this choice will be important  . There is one trade-off between bias and variance. 3.3.3 Variables to include. The above two subsections illustrate the key steps in matching procedure, and in this subsection, we briefly discuss what kinds of variables should be included in the matching, a.k.a feature selection, to improve the matching performance. Many literatures BIB007  BIB004 suggest to include as many variables that are related to the treatment assignment and the outcome as possible, in order to satisfy the strong ignorablity assumption. However, post-treatment variables, which are the variables affected by the treatment assignment, should be excluded in the matching procedure BIB002 . Moreover, besides the post-treatment variables, researchers also suggest excluding the instrumental variables  BIB013 , because they tend to amplify the bias of treatment effect estimator.