A Review on Quantification Learning <s> An Algorithm Based on EM <s> Let p1,…,pm be multivariate density functions and let α = (α1,…,αm)T be a proportion vector defining the mixture density , where and αi ≥ 0 for i = l,…,m. This paper investigates the problem of evaluation of the maximum-likelihood estimate of α. An acreage estimation application is presented using remotely sensed data. <s> BIB001 </s> A Review on Quantification Learning <s> An Algorithm Based on EM <s> Labeled data for classification could often be obtained by sampling that restricts or favors choice of certain classes. A classifier trained using such data will be biased, resulting in wrong inference and sub-optimal classification on new data. Given an unlabeled new data set we propose a bootstrap method to estimate its class probabilities by using an estimate of the classifier's accuracy on training data and an estimate of probabilities of classifier's predictions on new data. Then, we propose two methods to improve classification accuracy on new data. The first method can be applied only if a classifier was designed to predict posterior class probabilities where predictions of an existing classifier are adjusted according to the estimated class probabilities of new data. The second method can be applied to an arbitrary classification algorithm, but it requires retraining on the properly resampled data. The proposed bootstrap algorithm was validated through experiments with 500 replicates calculated on 1,000 realizations for each of 16 choices of data set size, number of classes, prior class probabilities and conditional probabilities describing a classifier's performance. Applications of the proposed methodology to a benchmark data set with various class probabilities on unlabeled data and balanced class probabilities on the training data provided strong evidence that the proposed methodology can be successfully used to significantly improve classification on unlabeled data. <s> BIB002 </s> A Review on Quantification Learning <s> An Algorithm Based on EM <s> It sometimes happens (for instance in case control studies) that a classifier is trained on a data set that does not reflect the true a priori probabilities of the target classes on real-world data. This may have a negative effect on the classification accuracy obtained on the real-world data set, especially when the classifier's decisions are based on the a posteriori probabilities of class membership. Indeed, in this case, the trained classifier provides estimates of the a posteriori probabilities that are not valid for this real-world data set (they rely on the a priori probabilities of the training set). Applying the classifier as is (without correcting its outputs with respect to these new conditions) on this new data set may thus be suboptimal. In this note, we present a simple iterative procedure for adjusting the outputs of the trained classifier with respect to these new a priori probabilities without having to refit the model, even when these probabilities are not known in advance. As a by-product, estimates of the new a priori probabilities are also obtained. This iterative algorithm is a straightforward instance of the expectation-maximization (EM) algorithm and is shown to maximize the likelihood of the new data. Thereafter, we discuss a statistical test that can be applied to decide if the a priori class probabilities have changed from the training set to the real-world data. The procedure is illustrated on different classification problems involving a multilayer neural network, and comparisons with a standard procedure for a priori probability estimation are provided. Our original method, based on the EM algorithm, is shown to be superior to the standard one for a priori probability estimation. Experimental results also indicate that the classifier with adjusted outputs always performs better than the original one in terms of classification accuracy, when the a priori probability conditions differ from the training set to the real-world data. The gain in classification accuracy can be significant. <s> BIB003 </s> A Review on Quantification Learning <s> An Algorithm Based on EM <s> How to forecast next year’s portfolio-wide credit default rate based on last year’s default observations and the current score distribution? A classical approach to this problem consists of fitting a mixture of the conditional score distributions observed last year to the current score distribution. This is a special (simple) case of a finite mixture model where the mixture components are fixed and only the weights of the components are estimated. The optimum weights provide a forecast of next year’s portfolio-wide default rate. We point out that the maximum-likelihood (ML) approach to fitting the mixture distribution not only gives an optimum but even an exact fit if we allow the mixture components to vary but keep their density ratio fixed. From this observation we can conclude that the standard default rate forecast based on last year’s conditional default rates will always be located between last year’s portfolio-wide default rate and the ML forecast for next year. As an application example, cost quantification is then discussed. We also discuss how the mixture model based estimation methods can be used to forecast total loss. This involves the reinterpretation of an individual classification problem as a collective quantification problem. <s> BIB004
This is probably the most popular algorithm of this group, at least in quantification articles. BIB003 introduce a classification method that is designed to deal with those situations in which the distribution of the classes change. The main goal is not to estimate the class distribution but to adjust the classifier according to the class distribution drift without the need of re-training it. A minor limitation of this approach is that it can only be applied to probabilistic classifiers. Interestingly, BIB001 studied this very same iterative approach, using densities instead of conditional probabilities for maximum likelihood estimate of mixture proportions. Given a training set D, the algorithm learns a probabilistic classifier that it is able to estimate P tr (y =+1|x ). Based on probability theory, the optimal Bayes classifier for the testing data can be modeled using the classifier obtained with the training data, so there is no need to re-train the classifier. The learning assumptions are the same than in other quantification methods, that is, P tr (y) P tst (y), but P tr (x |y) = P tst (x |y). Taking into account that P tr (y) P tst (y), the model for the testing set, P tst (y =+1|x ), should be different than that obtained from the training data, P tr (y =+1|x ). The aim is to obtain P tst (y =+1|x ) using P tr (y =+1|x ). Applying Bayes's theorem for the training and the testing distributions, we have that But given that P tr (x |y) = P tst (x |y), Finally, the optimal model for the testing distribution is The denominator ensures that a posteriori probabilities sum to 1. Notice that the a posteriori probabilities depend nonlinearly on the a priori probabilities. This expression implies that, even if the classifier is an optimal Bayesian model for the training set, it will not be optimal for the testing set when the a priori probabilities change. However, if we know the new a priori probabilities, then it is possible to recover an optimal model for the distribution of the testing data applying the Bayess rule. The final conclusion is that to update the classifier, we just need to estimate the class distribution in the testing data (P tst (y =+1), P tst (y = −1)). The solution proposed by BIB003 is based on the Expected-Maximization (EM) algorithm. Given a test set T = {x 1 , . . . , x m }, the goal is to maximize the likelihood: Since the within-class densities remain the same, we have just to estimate (P tst (y =+1), P tst (y = −1)) to maximize the likelihood. The authors propose to apply the EM algorithm arguing that there is not a closed-form solution to this problem. The steps of the algorithm are defined as follows: 74:24 P. González et al. Recently, Tasche (2017) presents a nice theoretical study about this EM approach. Based on the work by BIB001 that provided conditions for the convergence of the algorithm (it converges for the binary case), and on more recent articles, namely BIB004 and Du Plessis and  , the author proves that the EM algorithm finds the true values of the class prevalences under prior probability shift, that is, when P tr (y) P tst (y) but P tr (x |y) = P tst (x |y) and it is Fisher consistent (unbiased). BIB002 present an iterative procedure aimed at improving the classification accuracy. The method starts by training a classifier with the original training set that is applied to obtain an estimation of the a priori class probabilities using a bootstrap process. According to these probabilities, the training dataset D is sampled (with replacement) to obtain a new training set, D new , which is used for re-training the classifier. These two steps are repeated until convergence. The most interesting step from the point of view of quantification is the bootstrap process to estimate the class distribution. This process requires two sets: the labelled training dataset D (or D new ) and an unlabelled test set, T . As p j is the actual probability of class j in T , P (k, j) = P (h(x i ) = k |y i = j) is the probability of predicting the class k when the actual class is j andp k is the probability of predicting class k in T . Using the law of total probability,p k iŝ