In this work, we addressed the issue of applying a stochastic classifier and a local, fuzzy confusion matrix under the framework of multi-label classification. We proposed a novel solution to the problem of correcting label pairwise ensembles. The main step of the correction procedure is to compute classifier- specific competence and cross-competence measures, which estimates error pattern of the underlying classifier. We considered two improvements of the method of obtaining confusion matrices. The first one is aimed to deal with imbalanced labels. The other utilizes double labelled instances which are usually removed during the pairwise transformation. The proposed methods were evaluated using 29 benchmark datasets. In order to assess the efficiency of the introduced models, they were compared against 1 state-of-the-art approach and the correction scheme based on the original method of confusion matrix estimation. The comparison was performed using four different multi-label evaluation measures: macro and micro-averaged F1 loss, zero-one loss and Hamming loss. Additionally, we investigated relations between classification quality, which is expressed in terms of different quality criteria, and characteristics of multi-label datasets such as average imbalance ratio or label density. The experimental study reveals that the correction approaches significantly outperforms the reference method only in terms of zero-one loss.
Another technique of decomposition of multi-label classification task into a set of binary classifiers is the scheme. Under this framework, a binary classifier is trained for each pair of labels, and its task is to separate given labels. The outcome of the classifier is interpreted as an expression of pairwise preference in a label ranking @cite_13 . In other words, the classification outcome shows which label is preferred within the pair. Finally, outputs of binary models are collected and a final ranking is formed using chosen merging procedure @cite_1 . To convert the ranking into a binary response a thresholding procedure must be employed @cite_13 .
@cite_1 @cite_13
2102705755 1990016442
Preference learning is an emerging topic that appears in different guises in the recent literature. This work focuses on a particular learning scenario called label ranking, where the problem is to learn a mapping from instances to rankings over a finite number of labels. Our approach for learning such a mapping, called ranking by pairwise comparison (RPC), first induces a binary preference relation from suitable training data using a natural extension of pairwise classification. A ranking is then derived from the preference relation thus obtained by means of a ranking procedure, whereby different ranking methods can be used for minimizing different loss functions. In particular, we show that a simple (weighted) voting strategy minimizes risk with respect to the well-known Spearman rank correlation. We compare RPC to existing label ranking methods, which are based on scoring individual labels instead of comparing pairs of labels. Both empirically and theoretically, it is shown that RPC is superior in terms of computational efficiency, and at least competitive in terms of accuracy. We study the problem of label ranking, a machine learning task that consists of inducing a mapping from instances to rankings over a finite number of labels. Our learning method, referred to as ranking by pairwise comparison (RPC), first induces pairwise order relations (preferences) from suitable training data, using a natural extension of so-called pairwise classification. A ranking is then derived from a set of such relations by means of a ranking procedure. In this paper, we first elaborate on a key advantage of such a decomposition, namely the fact that it allows the learner to adapt to different loss functions without re-training, by using different ranking procedures on the same predicted order relations. In this regard, we distinguish between two types of errors, called, respectively, ranking error and position error. Focusing on the position error, which has received less attention so far, we then propose a ranking procedure called ranking through iterated choice as well as an efficient pairwise implementation thereof. Apart from a theoretical justification of this procedure, we offer empirical evidence in favor of its superior performance as a risk minimizer for the position error.