A survey of multiple classifier systems as hybrid systems <s> Diversifying models <s> This paper reviews the supervised learning versions of the no-free-lunch theorems in a simplified form. It also discusses the significance of those theorems, and their relation to other aspects of supervised learning. <s> BIB001 </s> A survey of multiple classifier systems as hybrid systems <s> Diversifying models <s> The paper shows the possibilities of generalizing the two-class classification into multi-class classification by means of a fuzzy inference system. Fuzzy combiner harnesses the support values from classifiers to provide final response having no other restrictions on their structure. We compare proposed combination methods with ECOC and two variations of decision templates, based on Euclidean and symmetric distance. The effectiveness of the proposed combination method based on the fuzzy logic theory is also evaluated via computer experiments carried out on benchmark datasets. <s> BIB002
Ensembles with individual classifiers based on different classification models take advantage of the different biases of each classifier model BIB001 . However, the combination rule should be carefully chosen. We can combine the class labels but in the case of continuous outputs we have to normalize them, e.g., using fuzzy approach BIB002 . We could use the different versions of the same model as well, because many machine learning algorithms do not guarantee to find the optimal classifier. Combining the results of various initializations may give good results. Alternatively, a pool of classifiers can be produced by noise injection. Regarding neural networks  it is easy to train pools of networks where each of them is trained starting from randomly chosen initial weights. Regarding decisions tree we can choose randomly the test for a given node among the possible tests according to the value of a splitting criterion.