A Survey on Causal Inference <s> Advertising <s> We study decision making in environments where the reward is only partially observed, but can be modeled as a function of an action and an observed context. This setting, known as contextual bandits, encompasses a wide variety of applications including health-care policy and Internet advertising. A central task is evaluation of a new policy given historic data consisting of contexts, actions and received rewards. The key challenge is that the past data typically does not faithfully represent proportions of actions taken by a new policy. Previous approaches rely either on models of rewards or models of the past policy. The former are plagued by a large bias whereas the latter have a large variance. ::: In this work, we leverage the strength and overcome the weaknesses of the two approaches by applying the doubly robust technique to the problems of policy evaluation and optimization. We prove that this approach yields accurate value estimates when we have either a good (but not necessarily consistent) model of rewards or a good (but not necessarily consistent) model of past policy. Extensive empirical comparison demonstrates that the doubly robust approach uniformly improves over existing techniques, achieving both lower variance in value estimation and better policies. As such, we expect the doubly robust approach to become common practice. <s> BIB001 </s> A Survey on Causal Inference <s> Advertising <s> Advertising effectiveness measurement is a fundamental problem in online advertising. Various causal inference methods have been employed to measure the causal effects of ad treatments. However, existing methods mainly focus on linear logistic regression for uni-variate and binary treatments and are not well suited for complex ad treatments of multi-dimensions, where each dimension could be discrete or continuous. In this paper we propose a novel two-stage causal inference framework for assessing the impact of complex ad treatments. In the first stage, we estimate the propensity parameter via a sparse additive model; in the second stage, a propensity-adjusted regression model is applied for measuring the treatment effect. Our approach is shown to provide an unbiased estimation of the ad effectiveness under regularity conditions. To demonstrate the efficacy of our approach, we apply it to a real online advertising campaign to evaluate the impact of three ad treatments: ad frequency, ad channel, and ad size. We show that the ad frequency usually has a treatment effect cap when ads are showing on mobile device. In addition, the strategies for choosing best ad size are completely different for mobile ads and online ads. <s> BIB002 </s> A Survey on Causal Inference <s> Advertising <s> As the online advertising industry has evolved into an age of diverse ad formats and delivery channels, users are exposed to complex ad treatments involving various ad characteristics. The diversity and generality of ad treatments call for accurate and causal measurement of ad effectiveness, i.e., how the ad treatment causes the changes in outcomes without the confounding effect by user characteristics. Various causal inference approaches have been proposed to measure the causal effect of ad treatments. However, most existing causal inference methods focus on univariate and binary treatment and are not well suited for complex ad treatments. Moreover, to be practical in the data-rich online environment, the measurement needs to be highly general and efficient, which is not addressed in conventional causal inference approaches. In this paper we propose a novel causal inference framework for assessing the impact of general advertising treatments. Our new framework enables analysis on uni- or multi-dimensional ad treatments, where each dimension (ad treatment factor) could be discrete or continuous. We prove that our approach is able to provide an unbiased estimation of the ad effectiveness by controlling the confounding effect of user characteristics. The framework is computationally efficient by employing a tree structure that specifies the relationship between user characteristics and the corresponding ad treatment. This tree-based framework is robust to model misspecification and highly flexible with minimal manual tuning. To demonstrate the efficacy of our approach, we apply it to two advertising campaigns. In the first campaign we evaluate the impact of different ad frequencies, and in the second one we consider the synthetic ad effectiveness across TV and online platforms. Our framework successfully provides the causal impact of ads with different frequencies in both campaigns. Moreover, it shows that the ad frequency usually has a treatment effect cap, which is usually over-estimated by naive estimation. <s> BIB003 </s> A Survey on Causal Inference <s> Advertising <s> A widely used method for estimating counterfactuals and causal treatment effects from observational data is nearest-neighbor matching. This typically involves pairing each treated unit with its nearest-in-covariates control unit, and then estimating an average treatment effect from the set of matched pairs. Although straightforward to implement, this estimator is known to suffer from a bias that increases with the dimensionality of the covariate space, which can be undesirable in applications that involve high-dimensional data. To address this problem, we propose a novel estimator that first projects the data to a number of random linear subspaces, and it then estimates the median treatment effect by nearest-neighbor matching in each subspace. We empirically compute the mean square error of the proposed estimator using semi-synthetic data, and we demonstrate the method on real-world digital marketing campaign data. The results show marked improvement over baseline methods. <s> BIB004 </s> A Survey on Causal Inference <s> Advertising <s> When a new treatment is considered for use, whether a pharmaceutical drug or a search engine ranking algorithm, a typical question that arises is, will its performance exceed that of the current treatment? The conventional way to answer this counterfactual question is to estimate the effect of the new treatment in comparison to that of the conventional treatment by running a controlled, randomized experiment. While this approach theoretically ensures an unbiased estimator, it suffers from several drawbacks, including the difficulty in finding representative experimental populations as well as the cost of running randomized trials. Moreover, such trials neglect the huge quantities of available control-condition data, which in principle can be utilized for the harder task of predicting individualized effects. ::: ::: In this paper, we propose a discriminative framework for predicting the outcomes of a new treatment from a large dataset of the control condition and data from a small (and possibly unrepresentative) randomized trial comparing new and old treatments. Our learning objective, which requires minimal assumptions on the treatments, models the relation between the outcomes of the different conditions. This allows us to not only estimate mean effects but also to generate individual predictions for examples outside the small randomized sample. ::: ::: We demonstrate the utility of our approach through experiments in three areas: search engine operation, treatments to diabetes patients, and market value estimation of houses. Our results demonstrate that our approach can reduce the number and size of the currently performed randomized controlled experiments, thus saving significant time, money and effort on the part of practitioners. <s> BIB005 </s> A Survey on Causal Inference <s> Advertising <s> Propensity score matching and weighting are popular methods when estimating causal eects in observational studies. Beyond the assumption of unconfoundedness, however, these methods also require the model for the propensity score to be correctly specied. The recently proposed covariate balancing propensity score (CBPS) methodology increases the robustness to model misspecication by directly optimizing sample covariate balance between the treatment and control groups. In this paper, we extend the CBPS to a continuous treatment. We propose the covariate balancing generalized propensity score (CBGPS) methodology, which minimizes the association between covariates and the treatment. We develop both parametric and nonparametric approaches and show their superior performance over the standard maximum likelihood estimation in a simulation study. The CBGPS methodology is applied to an observational study, whose goal is to estimate the causal eects of political advertisements on campaign contributions. We also provide open-source software that implements the proposed methods. <s> BIB006 </s> A Survey on Causal Inference <s> Advertising <s> Click-through rate (CTR) prediction is the core problem of building advertising systems. Most existing state-of-the-art approaches model CTR prediction as binary classification problems, where displayed events with and without click feedbacks are respectively considered as positive and negative instances for training and offline validation. However, due to the selection mechanism applied in most advertising systems, a selection bias exists between distributions of displayed and non-displayed events. Conventional CTR models ignoring the bias may have inaccurate predictions and cause a loss of the revenue. To alleviate the bias, we need to conduct counterfactual learning by considering not only displayed events but also non-displayed events. In this paper, through a review of existing approaches of counterfactual learning, we point out some difficulties for applying these approaches for CTR prediction in a real-world advertising system. To overcome these difficulties, we propose a novel framework for counterfactual CTR prediction. In experiments, we compare our proposed framework against state-of-the-art conventional CTR models and existing counterfactual learning approaches. Experimental results show significant improvements. <s> BIB007
Properly measuring the effect of an advertising campaign can answer critical marketing questions such as whether a new advertisement increases the clicks, or whether a new campaign increases sales, etc. Since conducting randomized experiments is expensive and time-consuming, estimating the advertisement effect from the observational data is attracting increasing attention in both industry and research communities BIB002 BIB003 . In BIB004 , the randomized nearest neighbor matching method is proposed to estimate the treatment effect of digital marketing campaigns. In BIB006 , the covariate balancing generalized propensity score (CBGPS), discussed in Section 3.1.1, is applied to analyze the efficacy of political advertisements. However, in the online advertisement area, it is often required to deal with complex advertisement treatments, which could be a discrete or continuous, uni-or multi-dimensional treatment BIB002 . One advertisement is set as the baseline treatment, and the treatment effect is obtained by comparing the potential outcome of the treatment with different values and the baseline treatment. To estimate the potential outcome of treatment with multidimensional values, tree-based method BIB003 and sparse additive model based method BIB002 are proposed to enable the comparison between potential treatments and the baseline treatment. In addition to purely observational data, in the real-world scenarios, it is often the case that dataset is comprised of large samples from control condition(i.e., the old treatment) and small samples (possibility unrepresentative) from a randomized trial which contains both the control condition and the new treatment. In BIB005 , the small randomized trial dataset is connected with the large control dataset using the minimal set of modeling assumptions, which implies the models to predict the control and treated outcome to be similar. Under this assumption, the proposed method jointly learns the control and treated outcome predictor and regularizes the difference between the parameters of two predictors. The above discussions show the potential applications of the treatment effect estimation in decision evaluation: measuring the effect of the advertisement campaign. Another important application is to handle the selection bias. Due to the existing selection mechanism in the advertising systems, there is a distribution discrepancy between the displayed and non-displayed events BIB007 . Ignoring such bias would make the advertisement click prediction inaccurate, which would cause a loss of revenue. To handle the selection bias, similar to the doubly robust estimation mentioned in Section 3.1.1, doubly robust policy learning is proposed in BIB001 . It contains two sub-estimators: direct method estimator obtained from the observed samples, and IPS estimator with the propensity score as the sample weight. Furthermore, some works notice the difficulty of propensity score estimation due to the deterministic advertisement display policy in commercial advertisement systems. If the display policy is stochastic, the advertisements with low propensity scores still have a chance to appear in the observational dataset so that IPS can correct the selection bias. However, when the display policy is deterministic, the advertisements with low propensity scores are always absent in the observation, which makes propensity score estimation failed. This challenges motives the work of propensity-free doubly robust method proposed in BIB007 which improves the original doubly robust method in two folds: (1) Train the direct method on a small but unbiased data obtained under the uniform policy, which, to a certain degree, prevents the selection bias propagating to the non-displayed advertisements. (2) Avoid the propensity score estimation by setting the propensity score of the observed items as 1 and combines IPS with the direct method. In a nutshell, this propensity score free method relies on the direct method trained on a small unbiased dataset to give an unbiased prediction of the advertisement click. Apart from the applications discussed above, another important application is the advertisement recommendation, which is merged into the next subsection.