A review of single-source unsupervised domain adaptation <s> INTRODUCTION <s> Abstract : This paper reviews work on the effectiveness of different methods of matched sampling and statistical adjustment, alone and in combination, in reducing bias due to confounding x-variables when comparing two populations. The adjustment methods were linear regression adjustment for x continuous and direct standardization for x categorical. <s> BIB001 </s> A review of single-source unsupervised domain adaptation <s> INTRODUCTION <s> In this paper, I present a simple characterization of the sample selection bias problem that is also applicable to the conceptually distinct econometric problems that arise from truncated samples and from models with limited dependent variables. The problem of sample selection bias is fit within the conventional specification error framework of Griliches and Theil. A simple estimator is discussed that enables analysts to utilize ordinary regression methods to estimate models free of selection bias. The techniques discussed here are applied to re-estimate and test a model of female labor supply developed by the author. (1974). This paper is in three parts. In the first section, selection bias is presented within the specification error framework. In this section, general distributional assumptions are maintained. In section two, specific results are presented for the case of normal regression disturbances. Simple estimators are proposed and discussed. In the third section, empirical results are presented. <s> BIB002 </s> A review of single-source unsupervised domain adaptation <s> INTRODUCTION <s> This article addresses the issue of specification of econometric selectivity models and suggests approaches for the correction of selectivity bias. Our approaches provide ways to specify selectivity models without the assumption of multinormal distribution. Some flexible function forms for the correction of selectivity bias in the regression equation are derived. All the models considered can be estimated by simple consistent two stage methods. Our approaches provide simple procedures for the testing of selectivity bias without imposing restrictive distributional assumptions and also tests for the normality assumption. <s> BIB003 </s> A review of single-source unsupervised domain adaptation <s> INTRODUCTION <s> Preface.PART I: OVERVIEW AND BASIC APPROACHES.Introduction.Missing Data in Experiments.Complete-Case and Available-Case Analysis, Including Weighting Methods.Single Imputation Methods.Estimation of Imputation Uncertainty.PART II: LIKELIHOOD-BASED APPROACHES TO THE ANALYSIS OF MISSING DATA.Theory of Inference Based on the Likelihood Function.Methods Based on Factoring the Likelihood, Ignoring the Missing-Data Mechanism.Maximum Likelihood for General Patterns of Missing Data: Introduction and Theory with Ignorable Nonresponse.Large-Sample Inference Based on Maximum Likelihood Estimates.Bayes and Multiple Imputation.PART III: LIKELIHOOD-BASED APPROACHES TO THE ANALYSIS OF MISSING DATA: APPLICATIONS TO SOME COMMON MODELS.Multivariate Normal Examples, Ignoring the Missing-Data Mechanism.Models for Robust Estimation.Models for Partially Classified Contingency Tables, Ignoring the Missing-Data Mechanism.Mixed Normal and Nonnormal Data with Missing Values, Ignoring the Missing-Data Mechanism.Nonignorable Missing-Data Models.References.Author Index.Subject Index. <s> BIB004 </s> A review of single-source unsupervised domain adaptation <s> INTRODUCTION <s> Classifier learning methods commonly assume that the training data consist of randomly drawn examples from the same distribution as the test examples about which the learned model is expected to make predictions. In many practical situations, however, this assumption is violated, in a problem known in econometrics as sample selection bias. In this paper, we formalize the sample selection bias problem in machine learning terms and study analytically and experimentally how a number of well-known classifier learning methods are affected by it. We also present a bias correction method that is particularly useful for classifier evaluation under sample selection bias. <s> BIB005 </s> A review of single-source unsupervised domain adaptation <s> INTRODUCTION <s> We present a series of new theoretical, algorithmic, and empirical results for domain adaptation and sample bias correction in regression. We prove that the discrepancy is a distance for the squared loss when the hypothesis set is the reproducing kernel Hilbert space induced by a universal kernel such as the Gaussian kernel. We give new pointwise loss guarantees based on the discrepancy of the empirical source and target distributions for the general class of kernel-based regularization algorithms. These bounds have a simpler form than previous results and hold for a broader class of convex loss functions not necessarily differentiable, including L"q losses and the hinge loss. We also give finer bounds based on the discrepancy and a weighted feature discrepancy parameter. We extend the discrepancy minimization adaptation algorithm to the more significant case where kernels are used and show that the problem can be cast as an SDP similar to the one in the feature space. We also show that techniques from smooth optimization can be used to derive an efficient algorithm for solving such SDPs even for very high-dimensional feature spaces and large samples. We have implemented this algorithm and report the results of experiments both with artificial and real-world data sets demonstrating its benefits both for general scenario of adaptation and the more specific scenario of sample bias correction. Our results show that it can scale to large data sets of tens of thousands or more points and demonstrate its performance improvement benefits. <s> BIB006
G ENERALIZATION is the process of observing a finite number of samples and making statements about all possible samples. In the case of machine learning and pattern recognition, samples are used to train classifiers to make predictions for future samples. However, if the observed samples are not an accurate reflection of the underlying distribution, then the system will not generalize well to new samples. Data is biased if certain outcomes are systematically observed more or less frequently than they would for a uniformly-at-random sample. For example, a local patient population at a hospital differs from the global population along variables such as diet. As such, clinical data collected from that hospital is biased with respect to the global population. Statisticians have long studied biased samples under the term sample selection bias BIB002 , BIB003 ,  . Many correction procedures have been developed, most of which focus on the mechanism behind how the sample was generated. By estimating or -in cases where there is control of the experimental designknowing the probability that an instance will be observed or selected as part of the sample, the influence of the selection bias on the outcome variable can be controlled BIB001 , BIB004 ,  . However, many modern data collection procedures, such as internet crawlers, are less structured than, for example, sampling from patients in a hospital. It is therefore difficult, if not impossible, to estimate how the biased sample differs from the general population. But, in some settings, it is not necessary to generalize to the whole population. It might be more important to generalize towards a specific target subpopulation. For example, can data collected in European hospitals be used to train an intelligent prognosis system for hospitals in Africa? In order to target specific subpopulations, or domains, at least some information on them is necessary. Collecting unlabeled data from the target domain is usually possible, but labeling those is more difficult. Nonetheless, the unlabeled data gives an indication of how the two domains differ from each other. This information can be exploited to make the classifier adapt, i.e. change its decisions such that it generalizes better towards the target domain. The important questions to ask are: when and how can a classifier learn from a source domain and generalize to a target domain? These are the questions that we review in the current work. As for when, we consider the most notable assumptions and conditions from which generalization error bounds can be derived. Some of these conditions can be exploited to design adaptive classifiers. As for how, we present a categorization of current methods into three main categories. First, there are sample-based methods: these are based on correcting for biases in the data sampling procedure. Some instances have been observed too often or not enough, and misrepresent the situation. Examples include importance-weighting under covariate shift BIB006 , selection probability estimation in statistics BIB005 , and inverse propensity scoring in causal inference  . Secondly, there are feature-based methods, where source data is transformed to match target data. For example, an image can be brightened to account for differences in lighting conditions between two data sets. Thirdly, we consider what we call inference-based approaches. These methods consist of alternative parameter estimators. For instance, one could assume that the change in data distribution is due to an adversary and subsequently force the classifier to behave more conservatively. Or one could treat the source domain as prior knowledge that can be informative to clasification in the target domain. Note that the above categorization is not necessarily mutually exclusive.