A Review on Quantification Learning <s> Other Quantification Problems <s> This paper promotes a new task for supervised machine learning research: quantification - the pursuit of learning methods for accurately estimating the class distribution of a test set, with no concern for predictions on individual cases. A variant for cost quantification addresses the need to total up costs according to categories predicted by imperfect classifiers. These tasks cover a large and important family of applications that measure trends over time.The paper establishes a research methodology, and uses it to evaluate several proposed methods that involve selecting the classification threshold in a way that would spoil the accuracy of individual classifications. In empirical tests, Median Sweep methods show outstanding ability to estimate the class distribution, despite wide disparity in testing and training conditions. The paper addresses shifting class priors and costs, but not concept drift in general. <s> BIB001 </s> A Review on Quantification Learning <s> Other Quantification Problems <s> Many business applications track changes over time, for example, measuring the monthly prevalence of influenza incidents. In situations where a classifier is needed to identify the relevant incidents, imperfect classification accuracy can cause substantial bias in estimating class prevalence. The paper defines two research challenges for machine learning. The `quantification' task is to accurately estimate the number of positive cases (or class distribution) in a test set, using a training set that may have a substantially different distribution. The `cost quantification' variant estimates the total cost associated with the positive class, where each case is tagged with a cost attribute, such as the expense to resolve the case. Quantification has a very different utility model from traditional classification research. For both forms of quantification, the paper describes a variety of methods and evaluates them with a suitable methodology, revealing which methods give reliable estimates when training data is scarce, the testing class distribution differs widely from training, and the positive class is rare, e.g., 1% positives. These strengths can make quantification practical for business use, even where classification accuracy is poor. <s> BIB002 </s> A Review on Quantification Learning <s> Other Quantification Problems <s> The problem of estimating the class distribution (or prevalence) for a new unlabelled dataset (from a possibly different distribution) is a very common problem which has been addressed in one way or another in the past decades. This problem has been recently reconsidered as a new task in data mining, renamed quantification when the estimation is performed as an aggregation (and possible adjustment) of a single-instance supervised model (e.g., a classifier). However, the study of quantification has been limited to classification, while it is clear that this problem also appears, perhaps even more frequently, with other predictive problems, such as regression. In this case, the goal is to determine a distribution or an aggregated indicator of the output variable for a new unlabelled dataset. In this paper, we introduce a comprehensive new taxonomy of quantification tasks, distinguishing between the estimation of the whole distribution and the estimation of some indicators (summary statistics), for both classification and regression. This distinction is especially useful for regression, since predictions are numerical values that can be aggregated in many different ways, as in multi-dimensional hierarchical data warehouses. We focus on aggregative quantification for regression and see that the approaches borrowed from classification do not work. We present several techniques based on segmentation which are able to produce accurate estimations of the expected value and the distribution of the output variable. We show experimentally that these methods especially excel for the relevant scenarios where training and test distributions dramatically differ. <s> BIB003 </s> A Review on Quantification Learning <s> Other Quantification Problems <s> In recent years there has been a growing interest in text quantification, a supervised learning task where the goal is to accurately estimate, in an unlabelled set of items, the prevalence (or "relative frequency") of each class c in a predefined set C. Text quantification has several applications, and is a dominant concern in fields such as market research, the social sciences, political science, and epidemiology. In this paper we tackle, for the first time, the problem of ordinal text quantification, defined as the task of performing text quantification when a total order is defined on the set of classes; estimating the prevalence of "five stars" reviews in a set of reviews of a given product, and monitoring this prevalence across time, is an example application. We present OQT, a novel tree-based OQ algorithm, and discuss experimental results obtained on a dataset of tweets classified according to sentiment strength. <s> BIB004
Although the study of quantification has been mainly focused on quantification for classification domains, quantification also appears in other kind of learning problems such as regression, ordinal classification, cost-sensitive learning, and network quantification. BIB004 study the ordinal quantification tasks. The form of this learning problem is the same as multi-class quantification, Equation (2), but there is a total order (≺) defined on the set of classes { 1 , . . . , l } in ordinal quantification problems, that is, i ≺ j , ∀i, j : i < j. The goal is to predict the distribution of the ordered classes as accurately as possible. The interest of ordinal quantification is that it has many potential applications. For instance, training a model that learns how to quantify the number of consumers' opinions that fall inside a particular step in a predefined scale, for example, {VeryNegative, Negative, Fair, Positive, VeryPositive}. This kind of problem arises not only in e-commerce applications but also in many other contexts, especially those in which human preferences play a major role. Similarly to ordinal classification, ordinal quantification requires loss functions that take into account the order among the classes (see Section 4.3) and learning algorithms able to optimize such functions. Applying algorithms designed to deal with multi-class quantification tasks will lead to suboptimal models. Section 9 describes the algorithm proposed by BIB004 . Cost quantification (see Section 10) is the adaptation of cost-sensitive learning for quantification. Two different settings are considered for cost-sensitive learning in classification tasks. On the one hand, there are problems with class-dependent costs: C i, j is the cost of predicting the class i when the actual class is j . In the case of binary classification and assuming that C i,i = 0, the problem only requires two costs: C −,+ and C +,− representing the cost of misclassifying a positive example and a negative example, respectively. On the other hand, some problems have exampledependent costs: Each example has attached a cost attribute value, thus the training set has the form of D = {(x 1 , y 1 , c 1 ), . . . , (x m , y m , c m )}, where c i ∈ R + is the cost of resolving case x i . Only the latter setting has been considered for cost quantification with two different goals to estimate the total or average cost for each class. One example of application of cost quantification is to predict the cost of repairing a particular model of mobile phone over time. The classes in this problem are the breakdown types considered (e.g., cracked screens, battery faults), and we try to estimate the total or average cost of repairing all breakdowns that occurred over a period of time. The reason to use example-dependent costs is because the cost associated to each case can change depending on multiple factors (e.g., suppliers, labour costs). The first articles that deal with cost quantification were by BIB001 BIB002 . The author focuses on determining the total cost of positive examples on binary cost quantification tasks. However, most of the proposed methods in these articles estimate the average cost of the positive examples C + first and then multiply it by the prevalence of the positive class estimated using a binary quantifier. Multi-class cost quantification problems can be formally solved by learninḡ in which each element of the predicted vector, C j , represents the average cost associated to class j in the test sample. To obtain the total cost is as simple as computing |T | l j=1p j C j . Another possible real application of multi-class cost quantification is in the automatic plankton recognition domain. In this field, researchers are sometimes not interested in the distribution of the plankton species in a sample but in total biomass weight of each of the species. The problem is defined by assigning an individual cost to each training example (its biomass) and the model must be able to predict the total biomass of each species. BIB003 focus on quantification for regression problems (see Section 11) and propose several novel techniques based on discretization. In this case, the examples in the training set D have attached a real value, y i ∈ Y = R. The authors distinguish between two types of regression quantification tasks: (i) the estimation of the expected value for the given sample and (ii) the estimation of the whole distribution. One of the running examples of the article deals with a birthing center that collects data about each mother, including her medical history, and y is the baby weight at birth. With this training data, we may train a model to predict the average weight of the births that will take place for the current set of pregnant women that the center is monitoring for future deliveries. This is an example of the first group of tasks that can be formally described as The model predicts a single indicator that represents the expected value of the output (the mean). In some cases, this value can be supplemented with another value that expresses the confidence of the such prediction (e.g., using the standard error). However, other applications need to estimate the distribution of the output value. For instance, the birthing center may require a regression quantification model able to quantify how many lowweight births (weight lower than 2,500g) will take place. This corresponds to the second group of tasks, because we want to estimate a tail of the distribution. Formally, this problem can be represented in the following way:h : Given a test sample T and a value v,h(T , v) = P (ŷ i < v |x i ∈ T ), in whichŷ i is the predicted value for x i . Figure 1 depicts an example of quantification for regression. The graph on the left represents the actual distribution, while the graph on the right shows an estimation of the whole distribution and the expected value represented with a vertical line (the mean). In all previous quantification applications described before, data are represented in the conventional format of attribute-value pairs. But with the growth of social networks, many applications demand network quantification methods able to deal with data presented in relational networks. Networks offer additional information, as the relationships between the nodes of the network (e.g., friendship or following/follower relations in social networks) that can be exploited to model the collective behavior. While collective classification seeks to classify the interlinked objects of the 74:6 P. González et al. network, the goal of network quantification is to quantify them. Network data can be represented as an undirected graph G (V , E, L ), in which the network G is composed by a set of nodes or vertices V , a set of edges, E, and a set of classes, L . The objective of the methods described in Section 12 is to exploit the information in G to obtain better prevalence estimates for the classes in L .