A Review on Quantification Learning <s> Regress & Sum, Regress & Splice <s> Many business applications track changes over time, for example, measuring the monthly prevalence of influenza incidents. In situations where a classifier is needed to identify the relevant incidents, imperfect classification accuracy can cause substantial bias in estimating class prevalence. The paper defines two research challenges for machine learning. The `quantification' task is to accurately estimate the number of positive cases (or class distribution) in a test set, using a training set that may have a substantially different distribution. The `cost quantification' variant estimates the total cost associated with the positive class, where each case is tagged with a cost attribute, such as the expense to resolve the case. Quantification has a very different utility model from traditional classification research. For both forms of quantification, the paper describes a variety of methods and evaluates them with a suitable methodology, revealing which methods give reliable estimates when training data is scarce, the testing class distribution differs widely from training, and the positive class is rare, e.g., 1% positives. These strengths can make quantification practical for business use, even where classification accuracy is poor. <s> BIB001 </s> A Review on Quantification Learning <s> Regress & Sum, Regress & Splice <s> The problem of estimating the class distribution (or prevalence) for a new unlabelled dataset (from a possibly different distribution) is a very common problem which has been addressed in one way or another in the past decades. This problem has been recently reconsidered as a new task in data mining, renamed quantification when the estimation is performed as an aggregation (and possible adjustment) of a single-instance supervised model (e.g., a classifier). However, the study of quantification has been limited to classification, while it is clear that this problem also appears, perhaps even more frequently, with other predictive problems, such as regression. In this case, the goal is to determine a distribution or an aggregated indicator of the output variable for a new unlabelled dataset. In this paper, we introduce a comprehensive new taxonomy of quantification tasks, distinguishing between the estimation of the whole distribution and the estimation of some indicators (summary statistics), for both classification and regression. This distinction is especially useful for regression, since predictions are numerical values that can be aggregated in many different ways, as in multi-dimensional hierarchical data warehouses. We focus on aggregative quantification for regression and see that the approaches borrowed from classification do not work. We present several techniques based on segmentation which are able to produce accurate estimations of the expected value and the distribution of the output variable. We show experimentally that these methods especially excel for the relevant scenarios where training and test distributions dramatically differ. <s> BIB002
The first method studied was in fact introduced (and discarded) by BIB001 for cost quantification and it is the counterpart of Classify & Count for classification and Classify & Total for cost quantification. The Regress & Sum (RS) method first trains a regression model, using its predictions over the test set to compute the mean,ρ RS presents the same problems as CC and CT for their respective learning tasks. It mainly depends on the precision of the regression model. The same method takes a different name, Regress & Splice (RS), when it is used to make predictions about the whole distribution. Both methods have deliberately the same acronym, because they are in fact the same algorithm able to predict a single indicator (Sum) or the distribution (Splice). In the latter case, RS predicts the probability that the output for a random example from the test set, T , is less than a given value v: However, using this equation directly with predictionsŷ i obtained from the regressor produces estimated distributions very compact (low dispersion) with a highly peaked shape. The authors solve this issue, which is even more severe for the methods based on discretization (Section 11.2), alteringŷ i by a random (normal) jitter: in which σ Y is the standard deviation of the set of predictions of test set T and rnorm 0,1 is a random number generated from a typified normal distribution. This equation is applied for all the methods described here in the experiments reported in BIB002 . Based on the RS method, BIB002 introduce the Adjusted Regress & Sum (ARS) method. Following a similar idea that AC and PAC methods, ARS adjusts the prediction of RS. First, ARS computes the Quantification Error (QE) of the regression over the training set D: This value is used to adjust RS using the following expression: The degree of the adjustment depends on the value of the hyper-parameter α, and it can be seen as a way to calibrate the prediction made by the RS method. The same expression can be used to estimate the whole distribution, applied to Equation (78).