Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> SUMMARY A general class of regression models for ordinal data is developed and discussed. These models utilize the ordinal nature of the data by describing various modes of stochastic ordering and this eliminates the need for assigning scores or otherwise assuming cardinality instead of ordinality. Two models in particular, the proportional odds and the proportional hazards models are likely to be most useful in practice because of the simplicity of their interpretation. These linear models are shown to be multivariate extensions of generalized linear models. Extensions to non-linear models are discussed and it is shown that even here the method of iteratively reweighted least squares converges to the maximum likelihood estimate, a property which greatly simplifies the necessary computation. Applications are discussed with the aid of examples. <s> BIB001 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> A comprehensive look at learning and generalization theory. The statistical theory of learning and generalization concerns the problem of choosing desired functions on the basis of empirical data. Highly applicable to a variety of computer science and robotics fields, this book offers lucid coverage of the theory as a whole. Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more. <s> BIB002 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> In many classification problems the domains of the attributes and the classes are linearly orderded. For such problems the classification rule often needs to be order-preserving or monotone as we call it. Since the known decision tree methods generate non-monotone trees, these methods are not suitable for monotone classification problems. We provide an order-preserving tree-generation algorithm for multiattribute classification problems with k linearly ordered classes, and an algorithm for repairing non-monotone decision trees. The performance of these algorithms is tested on random monotone datasets. <s> BIB003 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> This paper is devoted to the problem of learning to predict ordinal (i.e., ordered discrete) classes using classification and regression trees. We start with S-CART, a tree induction algorithm, and study various ways of transforming it into a learner for ordinal classification tasks. These algorithm variants are compared on a number of benchmark data sets to verify the relative strengths and weaknesses of the strategies and to study the trade-off between optimal categorical classification accuracy (hit rate) and minimum distance-based error. Preliminary results indicate that this is a promising avenue towards algorithms that combine aspects of classification and regression. <s> BIB004 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> In many classification problems the domains of the attributes and the classes are linearly ordered. For such problems the classification rule often needs to be order-preserving or monotonic as we call it. Since the known decision tree methods generate non-monotonic trees, these methods are not suitable for monotonic classification problems. We provide an order-preserving tree-generation algorithm for multi-attribute classification problems with $k$ linearly ordered classes, and an algorithm for repairing non-monotonic decision trees. The performance of these algorithms is tested on a real-world financial dataset and on random monotonic datasets. <s> BIB005 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> Machine learning methods for classification problems commonly assume that the class values are unordered. However, in many practical applications the class values do exhibit a natural order--for example, when learning how to grade. The standard approach to ordinal classification converts the class value into a numeric quantity and applies a regression learner to the transformed data, translating the output back into a discrete class value in a post-processing step. A disadvantage of this method is that it can only be applied in conjunction with a regression scheme. ::: ::: In this paper we present a simple method that enables standard classification algorithms to make use of ordering information in class attributes. By applying it in conjunction with a decision tree learner we show that it outperforms the naive approach, which treats the class values as an unordered set. Compared to special-purpose algorithms for ordinal classification our method has the advantage that it can be applied without any modification to the underlying learning scheme. <s> BIB006 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> We discuss the problem of ranking k instances with the use of a "large margin" principle. We introduce two main approaches: the first is the "fixed margin" policy in which the margin of the closest neighboring classes is being maximized — which turns out to be a direct generalization of SVM to ranking learning. The second approach allows for k - 1 different margins where the sum of margins is maximized. This approach is shown to reduce to v-SVM when the number of classes k - 2. Both approaches are optimal in size of 2l where l is the total number of training examples. Experiments performed on visual classification and "collaborative filtering" show that both approaches outperform existing ordinal regression algorithms applied for ranking and multi-class SVM applied to general multi-class classification. <s> BIB007 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> Motivated by recent work studying massive imaging data in the neuroimaging literature, we propose multivariate varying coefficient models (MVCM) for modeling the relation between multiple functional responses and a set of covariates. We develop several statistical inference procedures for MVCM and systematically study their theoretical properties. We first establish the weak convergence of the local linear estimate of coefficient functions, as well as its asymptotic bias and variance, and then we derive asymptotic bias and mean integrated squared error of smoothed individual functions and their uniform convergence rate. We establish the uniform convergence rate of the estimated covariance function of the individual functions and its associated eigenvalue and eigenfunctions. We propose a global test for linear hypotheses of varying coefficient functions, and derive its asymptotic distribution under the null hypothesis. We also propose a simultaneous confidence band for each individual effect curve. We conduct Monte Carlo simulation to examine the finite-sample performance of the proposed procedures. We apply MVCM to investigate the development of white matter diffusivities along the genu tract of the corpus callosum in a clinical study of neurodevelopment. <s> BIB008 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> In this paper we present a simple to implement truly online large margin version of the Perceptron ranking (PRank) algorithm, called the OAP-BPM (Online Aggregate Prank-Bayes Point Machine) algorithm, which finds a rule that correctly ranks a given training sequence of instance and target rank pairs. PRank maintains a weight vector and a set of thresholds to define a ranking rule that maps each instance to its respective rank. The OAP-BPM algorithm is an extension of this algorithm by approximating the Bayes point, thus giving a good generalization performance. The Bayes point is approximated by averaging the weights and thresholds associated with several PRank algorithms run in parallel. In order to ensure diversity amongst the solutions of the PRank algorithms we randomly subsample the stream of incoming training examples. We also introduce two new online versions of Bagging and the voted Perceptron using the same randomization trick as OAP-BPM, hence are referred to as OAP with extension -Bagg and -VP respectively. A rank learning experiment was conducted on a synthetic data set and collaborative filtering experiments on a number of real world data sets were conducted, showing that OAP-BPM has a better performance compared to PRank and a pure online regression algorithm, albeit with a higher computational cost, though is not too prohibitive. <s> BIB009 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> This paper presents a method for learning a distance metric from relative comparison such as "A is closer to B than A is to C". Taking a Support Vector Machine (SVM) approach, we develop an algorithm that provides a flexible way of describing qualitative training data as a set of constraints. We show that such constraints lead to a convex quadratic programming problem that can be solved by adapting standard methods for SVM training. We empirically evaluate the performance and the modelling flexibility of the algorithm on a collection of text documents. <s> BIB010 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> We consider supervised learning of a ranking function, which is a mapping from instances to total orders over a set of labels (options). The training information consists of examples with partial (and possibly inconsistent) information about their associated rankings. From these, we induce a ranking function by reducing the original problem to a number of binary classification problems, one for each pair of labels. The main objective of this work is to investigate the trade-off between the quality of the induced ranking function and the computational complexity of the algorithm, both depending on the amount of preference information given for each example. To this end, we present theoretical results on the complexity of pairwise preference learning, and experimentally investigate the predictive performance of our method for different types of preference information, such as top-ranked labels and complete rankings. The domain of this study is the prediction of a rational agent's ranking of actions in an uncertain environment. <s> BIB011 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> In this paper, we propose a probabilistic kernel approach to preference learning based on Gaussian processes. A new likelihood function is proposed to capture the preference relations in the Bayesian framework. The generalized formulation is also applicable to tackle many multiclass problems. The overall approach has the advantages of Bayesian methods for model selection and probabilistic prediction. Experimental results compared against the constraint classification approach on several benchmark datasets verify the usefulness of this algorithm. <s> BIB012 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> Abstract The classification problem statement of multicriteria decision analysis is to model the classification of the alternatives/actions according to the decision maker's preferences. These models are based on outranking relations, utility functions or (linear) discriminant functions. Model parameters can be given explicitly or learnt from a preclassified set of alternatives/actions. In this paper we propose a novel approach, the Continuous Decision (CD) method, to learn parameters of a discriminant function, and we also introduce its extension, the Continuous Decision Tree (CDT) method, which describes the classification more accurately. The proposed methods are results of integration of Machine Learning methods in Decision Analysis. From a Machine Learning point of view, the CDT method can be considered as an extension of the C4.5 decision tree building algorithm that handles only numeric criteria but applies more complex tests in the inner nodes of the tree. For the sake of easier interpretation, the decision trees are transformed to rules. <s> BIB013 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> This work is inspired by the so-called reranking tasks in natural language processing. In this paper, we first study the ranking, reranking, and ordinal regression algorithms proposed recently in the context of ranks and margins. Then we propose a general framework for ranking and reranking, and introduce a series of variants of the perceptron algorithm for ranking and reranking in the new framework. Compared to the approach of using pairwise objects as training samples, the new algorithms reduces the data complexity and training time. We apply the new perceptron algorithms to the parse reranking and machine translation reranking tasks, and study the performance of reranking by employing various definitions of the margins. <s> BIB014 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> Ordinal regression has become an effective way of learning user preferences, but most research focuses on single regression problems. In this paper we introduce collaborative ordinal regression, where multiple ordinal regression tasks are handled simultaneously. Rather than modeling each task individually, we explore the dependency between ranking functions through a hierarchical Bayesian model and assign a common Gaussian Process (GP) prior to all individual functions. Empirical studies show that our collaborative model outperforms the individual counterpart in preference learning applications. <s> BIB015 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> Classification of ordinal data is one of the most important tasks of relation learning. This paper introduces a new machine learning paradigm specifically intended for classification problems where the classes have a natural order. The technique reduces the problem of classifying ordered classes to the standard two-class problem. The introduced method is then mapped into support vector machines and neural networks. Generalization bounds of the proposed ordinal classifier are also provided. An experimental study with artificial and real data sets, including an application to gene expression analysis, verifies the usefulness of the proposed approach. <s> BIB016 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> Nowadays the area under the receiver operating characteristics (ROC) curve, which corresponds to the Wilcoxon-Mann-Whitney test statistic, is increasingly used as a performance measure for binary classification systems. In this article we present a natural generalization of this concept for more than two ordered categories, a setting known as ordinal regression. Our extension of the Wilcoxon-Mann-Whitney statistic now corresponds to the volume under an r-dimensional surface (VUS) for r ordered categories and differs from extensions recently proposed for multi-class classification. VUS rather evaluates the ranking returned by an ordinal regression model instead of measuring the error rate, a way of thinking which has especially advantages with skew class or cost distributions. We give theoretical and experimental evidence of the advantages and different behavior of VUS compared to error rate, mean absolute error and other ranking-based performance measures for ordinal regression. The results demonstrate that the models produced by ordinal regression algorithms minimizing the error rate or a preference learning based loss, not necessarily impose a good ranking on the data. <s> BIB017 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> Ordinal regression is an important type of learning, which has properties of both classification and regression. Here we describe an effective approach to adapt a traditional neural network to learn ordinal categories. Our approach is a generalization of the perceptron method for ordinal regression. On several benchmark datasets, our method (NNRank) outperforms a neural network classification method. Compared with the ordinal regression methods using Gaussian processes and support vector machines, NNRank achieves comparable performance. Moreover, NNRank has the advantages of traditional neural networks: learning in both online and batch modes, handling very large training datasets, and making rapid predictions. These features make NNRank a useful and complementary tool for large-scale data mining tasks such as information retrieval, Web page ranking, collaborative filtering, and protein ranking in bioinformatics. The neural network software is available at: http://www.cs.missouri.edu/~chengji/cheng software.html. <s> BIB018 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> We consider the problem of learning dissimilarities between points via formulations which preserve a specified ordering between points rather than the numerical values of the dissimilarities. Dissimilarity ranking (d-ranking) learns from instances like "A is more similar to B than C is to D" or "The distance between E and F is larger than that between G and H". Three formulations of d-ranking problems are presented and new algorithms are presented for two of them, one by semidefinite programming (SDP) and one by quadratic programming (QP). Among the novel capabilities of these approaches are out-of-sample prediction and scalability to large problems. <s> BIB019 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> In many application areas of machine learning, prior knowledge concerning the monotonicity of relations between the response variable and predictor variables is readily available. Monotonicity may also be an important model requirement with a view toward explaining and justifying, decisions, Such as acceptance/rejection decisions. We propose a modified nearest neighbour algorithm for the construction of monotone classifiers from data. We start by making the training, data monotone with as few label changes as possible. The relabeled data set can he viewed as a monotone classifier that has the lowest possible error-rate on the training data. The relabeled data is subsequently used as the training sample by a modified nearest neighbour algorithm. This modified nearest neighbour rule produces predictions that are guaranteed to satisfy the monotonicity constraints. Hence, it is Much more likely to be accepted by the intended users. Our experiments show that monotone kNN often outperforms standard kNN in problems where the monotonicity constraints are applicable. <s> BIB020 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> We analyze the relationship between ordinal ranking and binary classification with a new technique called reverse reduction. In particular, we prove that the regret can be transformed between ordinal ranking and binary classification. The proof allows us to establish a general equivalence between the two in terms of hardness. Furthermore, we use the technique to design a novel boosting approach that improves any cost-sensitive base ordinal ranking algorithm. The approach extends the well-known AdaBoost to the area of ordinal ranking, and inherits many of its good properties. Experimental results demonstrate that our proposed approach can achieve decent training and test performance even when the base algorithm produces only simple decision stumps. <s> BIB021 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> The accuracy of k-nearest neighbor (kNN) classification depends significantly on the metric used to compute distances between different examples. In this paper, we show how to learn a Mahalanobis distance metric for kNN classification from labeled examples. The Mahalanobis metric can equivalently be viewed as a global linear transformation of the input space that precedes kNN classification using Euclidean distances. In our approach, the metric is trained with the goal that the k-nearest neighbors always belong to the same class while examples from different classes are separated by a large margin. As in support vector machines (SVMs), the margin criterion leads to a convex optimization based on the hinge loss. Unlike learning in SVMs, however, our approach requires no modification or extension for problems in multiway (as opposed to binary) classification. In our framework, the Mahalanobis distance metric is obtained as the solution to a semidefinite program. On several data sets of varying size and difficulty, we find that metrics trained in this way lead to significant improvements in kNN classification. Sometimes these results can be further improved by clustering the training examples and learning an individual metric within each cluster. We show how to learn and combine these local metrics in a globally integrated manner. <s> BIB022 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> Support vector machines (SVMs) were initially proposed to solve problems with two classes. Despite the myriad of schemes for multiclassification with SVMs proposed since then, little work has been done for the case where the classes are ordered. Usually one constructs a nominal classifier and a posteriori defines the order. The definition of an ordinal classifier leads to a better generalisation. Moreover, most of the techniques presented so far in the literature can generate ambiguous regions. All-at-Once methods have been proposed to solve this issue. In this work we devise a new SVM methodology based on the unimodal paradigm with the All-at-Once scheme for the ordinal classification. <s> BIB023 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> Ordinal regression has wide applications in many domains where the human evaluation plays a major role. Most current ordinal regression methods are based on Support Vector Machines (SVM) and suffer from the problems of ignoring the global information of the data and the high computational complexity. Linear Discriminant Analysis (LDA) and its kernel version, Kernel Discriminant Analysis (KDA), take into consideration the global information of the data together with the distribution of the classes for classification, but they have not been utilized for ordinal regression yet. In this paper, we propose a novel regression method by extending the Kernel Discriminant Learning using a rank constraint. The proposed algorithm is very efficient since the computational complexity is significantly lower than other ordinal regression methods. We demonstrate experimentally that the proposed method is capable of preserving the rank of data classes in a projected data space. In comparison to other benchmark ordinal regression methods, the proposed method is competitive in accuracy. <s> BIB024 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> Ordinal classification is a form of multi-class classification where there is an inherent ordering between the classes, but not a meaningful numeric difference between them. Although conventional methods, designed for nominal classes or regression problems, can be used to solve the ordinal data problem, there are benefits in developing models specific to this kind of data. This paper introduces a new rationale to include the information about the order in the design of a classification model. The method encompasses the inclusion of consistency constraints between adjacent decision regions. A new decision tree and a new nearest neighbour algorithms are then designed under that rationale. An experimental study with artificial and real data sets verifies the usefulness of the proposed approach. <s> BIB025 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> While ordinal classification problems are common in many situations, induction of ordinal decision trees has not evolved significantly. Conventional trees for regression settings or nominal classification are commonly induced for ordinal classification problems. On the other hand a decision tree consistent with the ordinal setting is often desirable to aid decision making in such situations as credit rating. In this work we extend a recently proposed strategy based on constraints defined globally over the feature space. We propose a bootstrap technique to improve the accuracy of the baseline solution. Experiments in synthetic and real data show the benefits of our proposal. <s> BIB026 </s> Multicriteria Models for Learning Ordinal Data: A Literature Review <s> Fig. 7 <s> Although Optical Music Recognition (OMR) has been the focus of much research for decades, the processing of handwritten musical scores is not yet satisfactory. The efforts made to find robust symbol representations and learning methodologies have not found a similar quality in the learning of the dissimilarity concept. Simple Euclidean distances are often used to measure dissimilarity between different examples. However, such distances do not necessarily yield the best performance. In this paper, we propose to learn the best distance for the k-nearest neighbor (k-NN) classifier. The distance concept will be tuned both for the application domain and the adopted representation for the music symbols. The performance of the method is compared with the support vector machine (SVM) classifier using both real and synthetic music scores. The synthetic database includes four types of deformations inducing variability in the printed musical symbols which exist in handwritten music sheets. The work presented here can open new research paths towards a novel automatic musical symbols recognition module for handwritten scores. <s> BIB027
Inductive Learning encompasses on two major research topics: Regression and classi f ication. Both thrives on finding the best function that explains our data. The former renders the reasoning's on a continuous domain whereas the latter on a discrete (finite) domain. Each one is divided in other subtopics being their thoroughly analysis more appropriate for other textbooks    and here depicted just for context. Learning mechanisms that solve ordinal problems have been tackled with both regression and classification strategies. Albeit being fundamentally different, both ordinal regression and ordinal classification methods have thrived among the scientific community, e.g., BIB016 BIB023 BIB006  BIB004 BIB001 BIB007 BIB026 , to name a few. The first works that tried to solve the classification of ordinal data were based on generalised linear models, as the cumulative model BIB001 . Tutz BIB008 presents a generic formulation for semi-parametric models extending the additive models  . In the machine learning community, Frank&Hall BIB006 have introduced a simple process which permits to explore information order in classification problems, using conventional binary classifiers as can be depicted in Fig. 8 . In  it is applied the minimal structural risk principle BIB002 to derive a learning algorithm based in pairs of points. Another way to learn ordering relation is by using classical algorithms of classification or regression and mapping the results into an ordinal scale. Kramer et al. BIB004 investigate the use of a learning algorithm for regression tasks-more specifically, a regression tree learner-to solve ordinal classification problems. In this case each class needs to be mapped to a numeric value. Kramer et al. BIB004 compare several different methods for doing this. However, if the class attribute represents a truly ordinal quantity-which, by definition, cannot be represented as a number in a meaningful way-there is no principled way of devising an appropriate mapping and this procedure is necessarily ad hoc. Harrington BIB009 argues that these type of approaches have many drawbacks as 1) makes regression learners sensitive to rank representation than their ordering and 2) since classification algorithms ignore rank order treating them as classes, it will be required more training data. Consequently, Harrington BIB009 presents a perceptron algorithm where its goal it to find a perceptron weight vector w w w which successfully projects all the instances into the k classes subintervals defined by some thresholds. Moreover, existing methods incurring ordinal regression approaches fit data in general by a single rule defined by parts through K-1 thresholds BIB017 . This has a drawback since a mapping is required to convert ranks into real values or vice-versa. Hence, determining this mapping function is in general very difficult and makes regression learners more sensitive to rank value than their pairwise ordering. Some of the aforementioned drawbacks were avoided in Shashua and Levin BIB007 work where a generalised formulation of Support Vector Machines (SVMs) applied to ordinal data was proposed. However, such models can be too complex. Cardoso in BIB016 proposed a reduction technique to solve data ordinal problem classification using only one binary classifier. Following this idea, Lin et al. BIB021 explored the potential of solving ordinal problems through binary classification methods whereas Cheng et al. in BIB018 presented an adaptation of the Neural Networks (NN) towards ordinal problems. In  an order relation is incorporated among classes by imposing an unimodal distribution. This fundamental principle allowed to delve simpler Neural Networks (NNs) classifiers. The same rationale was instantiated to SVMs in BIB023 through the all-at-once strategy by solving a multiclass ordinal problem through a single optimisation process. Sun et al. in BIB024 proposed a Kernel Discriminant Analysis (KDA) for ordinal data. Even though authors argued that finding an optimal projection would result in better reasonable results, in doing so one would loose its relation to the original features. Hence, in the case of need for interpretable results, through the usage of such methods, one would be unable to understand the reason of the outcome given specific features. Metric learning is research subject that recently has been gaining increasingly attention, specially in the machine learning community BIB022   . The performance of all machine learning algorithms depends critically on the metric that is used over the input space. Some learning algorithms, such as K-means and k-nearest neighbours, require a metric that will reflect important relationships between each classes in data and will allow to discriminate instances belonging to one class from others BIB027 . Ouyang BIB019 BIB010 explored this subject in the ordinal problem. In BIB019 by assuming that closer instances in the input space should translate an order of relation, a metric distance is learn so that pairs of instances are closer than the remainder pairs. However, class label is discarded in this approach. Other approaches  BIB012  BIB015 ] consisted on probabilistic approaches based in Gaussian processes to learn models for the ordinal problem. In BIB015 a collaborative approach is delved towards better, not only in accuracy, but also in a context of collaborative preference learning. Regarding decision trees (DTs) for ordinal data, some works consider problems that are monotone, i.e., all attributes have ordered domains. Meaning, if x x x, z z z are data points such that x x x ≤ z z z (x i ≤ z i for each criteria i) then their classes should satisfy the condition f (x x x) ≤ f (z z z), where f (.) is the labeling function. Potharst BIB003 BIB005  proposes a method that induces a binary decision tree from a monotone dataset. Other methods were also proposed for non-monotone datasets (the most likely scenario in the presence of noise) where the resulting tree may be non-monotone. In this scenario, a fuzzy operator was used instead of a entropy function for performance measurement BIB013 . Works on k-nearest neighbour for ordinal data seems even scarcer. Besides the well-known adaptation of using the median as labelling rule instead of mode for the k labels, literature only presents a modified version of the standard k-NN for the construction of monotone classifiers from data BIB020 . Again, this work continues to be limited by the assumption of monotonocity in the input data. In general, the monotone constraint was overcame in BIB025 BIB026 . Arguing that ordinality could not be captured directly from the input space, but from the feature space, authors explored a re-labelling approach on the output decision space through a postprocessing optimisation procedure. From the works until now revised, one has encountered several methods that make use of different procedures from operations research field, and other proposals design their learning models so that multicriteria can be rendered in the learning phase. In this setting, multicriteria assessment is simply performed over a set of diverse unattached reasoning's which renders the desirable outcomes without a clear understanding of which criteria contributed most. To overcome this, De Smet et al.  developed a k-means clustering algorithm in a multicriteria decision analysis perspective. In this section we have reviewed several learning approaches for the resolution of the ordinal problem. In the end, it is obvious how increasingly this subject has been studied. The reasons can be due to the awareness of its transversal usability in a set of diverse applications. However, due to the background of many researchers, many have tried to solve this problem through regression, classification and ranking methodologies. The work of Furnkranz et al. BIB011  despite using a pairwise approach, compared ranking and classification principles in their proposals. As final remark, one must note how vastly such methods can be employed such it has been explored by Van Belle et al. BIB014  . In these works, different approaches have been delved towards ranking, ordinal and survival analysis problems. Even though authors performed strict assumptions on data to develop their models, such as monotone data, it still is a good example of the importance of this topic in the inductive learning field.