A survey of multiple classifier systems as hybrid systems <s> Trainable Fuser <s> Abstract Binary decision trees based on nonparametric statistical models of the data provide a solution to difficult decision problems where there are many classes and many available features related in a complex manner. Unfortunately, the technique requires a very large training set and is often limited by the size of the training set rather than by the discriminatory power of the features. This paper demonstrates that higher classification accuracies can be obtained from the same training set by using a combination of decision trees and by reaching a consensus using Dempster and Shafer's theory of evidence. <s> BIB001 </s> A survey of multiple classifier systems as hybrid systems <s> Trainable Fuser <s> This paper introduces stacked generalization, a scheme for minimizing the generalization error rate of one or more generalizers. Stacked generalization works by deducing the biases of the generalizer(s) with respect to a provided learning set. This deduction proceeds by generalizing in a second space whose inputs are (for example) the guesses of the original generalizers when taught with part of the learning set and trying to guess the rest of it, and whose output is (for example) the correct guess. When used with multiple generalizers, stacked generalization can be seen as a more sophisticated version of cross-validation, exploiting a strategy more sophisticated than cross-validation's crude winner-takes-all for combining the individual generalizers. When used with a single generalizer, stacked generalization is a scheme for estimating (and then correcting for) the error of a generalizer which has been trained on a particular learning set and then asked a particular question. After introducing stacked generalization and justifying its use, this paper presents two numerical experiments. The first demonstrates how stacked generalization improves upon a set of separate generalizers for the NETtalk task of translating text to phonemes. The second demonstrates how stacked generalization improves the performance of a single surface-fitter. With the other experimental evidence in the literature, the usual arguments supporting cross-validation, and the abstract justifications presented in this paper, the conclusion is that for almost any real-world generalization problem one should use some version of stacked generalization to minimize the generalization error rate. This paper ends by discussing some of the variations of stacked generalization, and how it touches on other fields like chaos theory. <s> BIB002 </s> A survey of multiple classifier systems as hybrid systems <s> Trainable Fuser <s> Neural network-based modeling often involves trying multiple networks with different architectures and training parameters in order to achieve acceptable model accuracy. Typically, one of the trained networks is chosen as best, while the rest are discarded. [Hashem and Schmeiser (1995)] proposed using optimal linear combinations of a number of trained neural networks instead of using a single best network. Combining the trained networks may help integrate the knowledge acquired by the components networks and thus improve model accuracy. In this paper, we extend the idea of optimal linear combinations (OLCs) of neural networks and discuss issues related to the generalization ability of the combined model. We then present two algorithms for selecting the component networks for the combination to improve the generalization ability of OLCs. Our experimental results demonstrate significant improvements in model accuracy, as a result of using OLCs, compared to using the apparent best network. Copyright 1997 Elsevier Science Ltd. <s> BIB003 </s> A survey of multiple classifier systems as hybrid systems <s> Trainable Fuser <s> For pattern recognition, when a single classifier cannot provide a decision which is 100 percent correct, multiple classifiers should be able to achieve higher accuracy. This is because group decisions are generally better than any individual's. Based on this concept, a method called the "Behavior-Knowledge Space Method" was developed, which can aggregate the decisions obtained from individual classifiers and derive the best final decisions from the statistical point of view. Experiments on 46451 samples of unconstrained handwritten numerals have shown that this method achieves very promising performances and outperforms voting, Bayesian, and Dempster-Shafer approaches. > <s> BIB004 </s> A survey of multiple classifier systems as hybrid systems <s> Trainable Fuser <s> Neural-network ensembles have been shown to be very accurate classification techniques. Previous work has shown that an effective ensemble should consist of networks that are not only highly correct, but ones that make their errors on different parts of the input space as well. Most existing techniques, however, only indirectly address the problem of creating such a set of networks. In this paper we present a technique called ADDEMUP that uses genetic algorithms to directly search for an accurate and diverse set of trained networks. ADDEMUP works by first creating an initial population, then uses genetic operators to continually create new networks, keeping the set of networks that are as accurate as possible while disagreeing with each other as much as possible. Experiments on three DNA problems show that ADDEMUP is able to generate a set of trained networks that is more accurate than several existing approaches. Experiments also show that ADDEMUP is able to effectively incorporate prior knowledge, if available, to improve the quality of its ensemble. <s> BIB005 </s> A survey of multiple classifier systems as hybrid systems <s> Trainable Fuser <s> Recently, multiple classifier systems (MCS) have been used for practical applications to improve classification accuracy. Self-generating neural networks (SGNN) are one of the suitable base-classifiers for MCS because of their simple setting and fast learning. However, the computation cost of the MCS increases in proportion to the number of SGNN. In this paper, we propose a novel optimization method for the structure of the SGNN in the MCS. We compare the optimized MCS with two sampling methods. Experiments have been conducted to compare the optimized MCS with an unoptimized MCS, the MCS based on C4.5, and k-nearest neighbor. The results show that the optimized MCS can improve its classification accuracy as well as reducing the computation cost. <s> BIB006 </s> A survey of multiple classifier systems as hybrid systems <s> Trainable Fuser <s> In various application areas of pattern recognition, combining multiple classifiers is regarded as a method for achieving a substantial gain in performance of systems. The paper presents a method for handwritten Chinese character recognition to combine multiple classifiers based on statistics. Fusion strategies are discussed for providing a basis for combining classifiers. These combination strategies are experimentally tested on an online handwritten Chinese character recognition system. In our experiments, other combination approaches are also involved for comparison. <s> BIB007 </s> A survey of multiple classifier systems as hybrid systems <s> Trainable Fuser <s> Abstract Individual classification models are recently challenged by combined pattern recognition systems, which often show better performance. In such systems the optimal set of classifiers is first selected and then combined by a specific fusion method. For a small number of classifiers optimal ensembles can be found exhaustively, but the burden of exponential complexity of such search limits its practical applicability for larger systems. As a result, simpler search algorithms and/or selection criteria are needed to reduce the complexity. This work provides a revision of the classifier selection methodology and evaluates the practical applicability of diversity measures in the context of combining classifiers by majority voting. A number of search algorithms are proposed and adjusted to work properly with a number of selection criteria including majority voting error and various diversity measures. Extensive experiments carried out with 15 classifiers on 27 datasets indicate inappropriateness of diversity measures used as selection criteria in favour of the direct combiner error based search. Furthermore, the results prompted a novel design of multiple classifier systems in which selection and fusion are recurrently applied to a population of best combinations of classifiers rather than the individual best. The improvement of the generalisation performance of such system is demonstrated experimentally. <s> BIB008 </s> A survey of multiple classifier systems as hybrid systems <s> Trainable Fuser <s> Using an ensemble of models often results in better performance than using a single “best” model. We present a new approach based on data envelopment analysis (DEA) for model combination. We prove that for two-class classification problems, DEA models identify the same convex hull as does the popular receiver operating characteristics (ROC) analysis used for model combination. We further develop two DEA-based methods to combine classifiers for the more general k-class classification problems. Our results demonstrate that the two methods outperform other benchmark methods and suggest that DEA can be a powerful tool for model combination. <s> BIB009 </s> A survey of multiple classifier systems as hybrid systems <s> Trainable Fuser <s> The Multiple Classifier Systems are nowadays one of the most promising directions in pattern recognition. There are many methods of decision making by the ensemble of classifiers. The most popular are methods that have their origin in vote method, where the decision of the common classifier is a combination of individual classifiers decisions. This work presents method of weighted classifiers combination and experimental results of proposed algorithm. <s> BIB010 </s> A survey of multiple classifier systems as hybrid systems <s> Trainable Fuser <s> Combining classifiers are nowadays one of the most promising direction in pattern recognition. There are many methods of decision making which could be used by the ensemble of classifiers. The most popular are methods that have their origin in voting, where the decision of the common classifier is a combination of individual classifiers' outputs, i.e. classifiers' responses (class numbers) or values of discriminants. This work presents classifiers' weights establish via evolutionary approach. We propose such method for classifier ensemble which could produce classifiers better than Oracle classifier what was shown in the previous works of author. <s> BIB011
Fuser weight selection can be treated as a specific learning process BIB003 BIB006 . Shlien BIB001 used Dempster and Shafer's theory to reach a consensus on the weights to combine decision trees. Wozniak BIB010 trained the fuser using perceptron-like learning, evolutionary algorithm BIB011 BIB007 . Zheng used data envelopment analysis BIB009 . Other fuser trainable methods may be strictly related to ensemble pruning methods, when authors use some heuristic search algorithm to select the classifier ensemble, as BIB008 BIB005 according to the chosen fuser. We have to mention the group of combination methods built from pools of heterogenous classifiers, i.e. using different classification models, such as stacking BIB002 . This method trains combination block using individual classifier outputs presented during classification of the whole training set. Most of the combination methods do not take into consideration possible relations among individual classifiers. Huang and Suen BIB004 proposed BehaviorKnowledge Space method which aggregates the individual classifiers decision on the basis of the statistical approach.