A system of nested dichotomies is a method of decomposing a multi-class problem into a collection of binary problems. Such a system recursively applies binary splits to divide the set of classes into two subsets, and trains a binary classifier for each split. Many methods have been proposed to perform this split, each with various advantages and disadvantages. In this paper, we present a simple, general method for improving the predictive performance of nested dichotomies produced by any subset selection techniques that employ randomness to construct the subsets. We provide a theoretical expectation for performance improvements, as well as empirical results showing that our method improves the root mean squared error of nested dichotomies, regardless of whether they are employed as an individual model or in an ensemble setting.
Multiclass problems, i.e., classification problems involving more than two classes, are a common scenario in supervised classification. An important approach to solve this type of problems consists in using binary classifiers repeated times; within this category we find nested dichotomies. However, most of the methods for building nested dichotomies use a random strategy, which does not guarantee finding a good one. In this work, we propose new non-random methods for building nested dichotomies, using the idea of reducing misclassification errors by separating in the higher levels those classes that are easier to separate; and, in the lower levels those classes that are more difficult to separate. In order to evaluate the performance of the proposed methods, we compare them against methods that randomly build nested dichotomies, using some datasets (with mixed data) taken from the UCI repository.