However, a theoretical understanding of the generalization properties of AdaBoost, that explains this behaviour, is still missing. Early theoretical results on the generalization error of AdaBoost and other classification algorithms were based on margin-theory [1]}, [2]} and entropy bounds. In high-dimensional situations where the dimension of the features and number of base classifiers is larger than the number of observations \(n\) , these become meaningless. Another approach to explain the success of AdaBoost and other boosting algorithms is based on regularization through early stopping [3]}, [4]}, [5]}. However, by their nature these bounds can not explain generalization performance when the number of iterations grows large and the empirical training error equals zero. In the population setting [6]} showed that the generalization risk of AdaBoost converges to the Bayes risk, but this does also not indicate any performance guarantees for finite data.
