In the literature, averaging and majority voting are the most commonly used classifier combination approaches where the ensemble output is calculated based on the (weighted) average of the base classifier outputs or their mostly voted prediction. Bagging [1]} is a special case of majority voting for which the base classifiers are trained on differently versions [2]} of the same data obtained by resampling. More complex combination rules include methods such as boosting [3]}, where the classifiers are trained sequentially to compensate for the weaknesses of those already selected and stacking [4]}, where the outputs of all classifiers are fed into a new model to generate the final prediction. Another commonly used ensemble technique is random forests [5]}, which are composed of multiple decision trees trained on bootstrapped training data with an additional step of feature-bootstrapping to allow for a random selection (with replacement) of features at each tree node. The final decision is based on the (weighted) average of the outputs or the majority vote of the individual tree decisions.
