Further results on the margin explanation of boosting: new algorithm and experiments
Understanding the empirical success of boosting algorithms is an important theoretical problem in machine learning. One of the most influential works is the margin theory, which provides a series of upper bounds for the generalization error of any voting classifier in terms of the margins of the training data. Recently an equilibrium margin (Emargin) bound which is sharper than previously well known margin bounds is proposed. In this paper, we conduct extensive experiments to test the Emargin theory. Specifically, we develop an efficient algorithm that, given a boosting classifier (or a voting classifier in general) learns a new voting classifier which usually has a smaller Emargin bound. We then compare the performances of the two classifiers and find that the new classifier often has smaller test errors, which agrees with what the Emargin theory predicts.
Science China Information Sciences