A Review on Quantification Learning <s> METHODS BASED ON DISTRIBUTION MATCHING <s> Receiver Operating Characteristics (ROC) graphs are a useful technique for organizing classifiers and visualizing their performance. ROC graphs are commonly used in medical decision making, and in recent years have been increasingly adopted in the machine learning and data mining research communities. Although ROC graphs are apparently simple, there are some common misconceptions and pitfalls when using them in practice. This article serves both as a tutorial introduction to ROC graphs and as a practical guide for using them in research. <s> BIB001 </s> A Review on Quantification Learning <s> METHODS BASED ON DISTRIBUTION MATCHING <s> We consider the scenario where training and test data are drawn from different distributions, commonly referred to as sample selection bias. Most algorithms for this setting try to first recover sampling distributions and then make appropriate corrections based on the distribution estimate. We present a nonparametric method which directly produces resampling weights without distribution estimation. Our method works by matching distributions between training and testing sets in feature space. Experimental results demonstrate that our method works well in practice. <s> BIB002 </s> A Review on Quantification Learning <s> METHODS BASED ON DISTRIBUTION MATCHING <s> Quantification is the machine learning task of estimating test-data class proportions that are not necessarily similar to those in training. Apart from its intrinsic value as an aggregate statistic, quantification output can also be used to optimize classifier probabilities, thereby increasing classification accuracy. We unify major quantification approaches under a constrained multi-variate regression framework, and use mathematical programming to estimate class proportions for different loss functions. With this modeling approach, we extend existing binary-only quantification approaches to multi-class settings as well. We empirically verify our unified framework by experimenting with several multi-class datasets including the Stanford Sentiment Treebank and CIFAR-10. <s> BIB003
This section describes several methods that are based on matching the training distribution and the testing distribution. These methods do not need to classify individual examples and count them. The idea of most of them is to adjust some parameter to modify the training distribution trying to fit the testing distribution. Typically, this parameter is the predicted class distribution ([p, 1 −p] in a binary problem). In other words, searching for the value ofp that makes a modified version of the training distribution most similar to the testing distribution. The effectiveness of this idea depends on a key constraint that is the support condition, that is, ∀x, P tr (x ) = 0 → P tst (x ) = 0 BIB002 ). Roughly speaking, this means that the training set must be richer than the test set; otherwise, these methods cannot match well both distributions for regions in which P tst (x ) 0, but P tr (x ) = 0. However, several articles described in this section do not make clear this assumption. These methods usually have two main components: (i) a mechanism to represent distributions or, more formally, a density estimation method and (ii) a metric or a criterion to compare two given distributions. In fact, BIB003 unifies several quantification methods under a common framework defined by a constrained multi-variate regression model for quantification defined as T is a representation of the testing distribution, D the training distribution, andp the class distribution. The goal is to obtain the values ofp that minimize the difference between both distributions given a loss function. Therefore, the framework comprises the same two elements enumerated before. At least three of the approaches described in this section can be derived from this framework: the Mixture Model (see Section 8.3), the methods based on the Hellinger Distance (see Section 8.4), and King and Lu's method (2008) (see Section 8.5). Following this same idea, there are algorithms that also estimate the class distribution but their final goal is not to accurately predict such estimate but to improve the classifier accuracy. There is a lot of literature on this line of research, see, for instance,  and BIB001 for some seminal works. The algorithms included here are designed for unsupervised domain adaptation problems. These kind of techniques are aimed at tackling those situations in which the domain changes, that is, when a drift in the distribution is expected or detected. In such cases, characterizing the new distribution can help to boost the robustness and precision of the classifier.