TITLE: A Dataset-Level Geometric Framework for Ensemble Classifiers 

ABSTRACT: Ensemble classifiers have been investigated by many in the artificial
intelligence and machine learning community. Majority voting and weighted
majority voting are two commonly used combination schemes in ensemble learning.
However, understanding of them is incomplete at best, with some properties even
misunderstood. In this paper, we present a group of properties of these two
schemes formally under a dataset-level geometric framework. Two key factors,
every component base classifier's performance and dissimilarity between each
pair of component classifiers are evaluated by the same metric - the Euclidean
distance. Consequently, ensembling becomes a deterministic problem and the
performance of an ensemble can be calculated directly by a formula. We prove
several theorems of interest and explain their implications for ensembles. In
particular, we compare and contrast the effect of the number of component
classifiers on these two types of ensemble schemes. Empirical investigation is
also conducted to verify the theoretical results when other metrics such as
accuracy are used. We believe that the results from this paper are very useful
for us to understand the fundamental properties of these two combination
schemes and the principles of ensemble classifiers in general. The results are
also helpful for us to investigate some issues in ensemble classifiers, such as
ensemble performance prediction, selecting a small number of base classifiers
to obtain efficient and effective ensembles.
Ensemble classifiers have been investigated by many in the artificial
intelligence and machine learning community. Majority voting and weighted
majority voting are two commonly used combination schemes in ensemble learning.
However, understanding of them is incomplete at best, with some properties even
misunderstood. In this paper, we present a group of properties of these two
schemes formally under a dataset-level geometric framework. Two key factors,
every component base classifier's performance and dissimilarity between each
pair of component classifiers are evaluated by the same metric - the Euclidean
distance. Consequently, ensembling becomes a deterministic problem and the
performance of an ensemble can be calculated directly by a formula. We prove
several theorems of interest and explain their implications for ensembles. In
particular, we compare and contrast the effect of the number of component
classifiers on these two types of ensemble schemes. Empirical investigation is
also conducted to verify the theoretical results when other metrics such as
accuracy are used. We believe that the results from this paper are very useful
for us to understand the fundamental properties of these two combination
schemes and the principles of ensemble classifiers in general. The results are
also helpful for us to investigate some issues in ensemble classifiers, such as
ensemble performance prediction, selecting a small number of base classifiers
to obtain efficient and effective ensembles.
A Dataset-Level Geometric Framework for Ensemble Classifiers
http://arxiv.org/abs/2106.08658v1