The AdaBoost algorithm has the superiority of resisting overfitting. Understanding the mysteries of this phenomena is a very fascinating fundamental theoretical problem. Many studies are devoted to explaining it from statistical view and margin theory. In this paper, we illustrate it from feature learning viewpoint, and propose the AdaBoost+SVM algorithm, which can explain the resistant to overfitting of AdaBoost directly and easily to understand. Firstly, we adopt the AdaBoost algorithm to learn the base classifiers. Then, instead of directly weighted combination the base classifiers, we regard them as features and input them to SVM classifier. With this, the new coefficient and bias can be obtained, which can be used to construct the final classifier. We explain the rationality of this and illustrate the theorem that when the dimension of these features increases, the performance of SVM would not be worse, which can explain the resistant to overfitting of AdaBoost.
@cite_23 were the first ones to use the margin theory to explain this overfitting phenomenon. Define @math as the margin for @math with respect to @math . Use @math to refer as the probability with respect to sample weight vector @math , and @math to denote the probability with respect to uniform distribution over the sample @math . They first proved the following theorem to bound the generalization error of each voting classifier:
@cite_23
2798600727 1644339822 1971118040 2110596924
This article offers a simplified approach to the distribution theory of randomly weighted averages or @math -means @math , for a sequence of i.i.d.random variables @math , and independent random weights @math with @math and @math . The collection of distributions of @math , indexed by distributions of @math , is shown to encode Kingman's partition structure derived from @math . For instance, if @math has Bernoulli @math distribution on @math , the @math th moment of @math is a polynomial function of @math which equals the probability generating function of the number @math of distinct values in a sample of size @math from @math : @math . This elementary identity illustrates a general moment formula for @math -means in terms of the partition structure associated with random samples from @math , first developed by Diaconis and Kemperman (1996) and Kerov (1998) in terms of random permutations. As shown by Tsilevich (1997) if the partition probabilities factorize in a way characteristic of the generalized Ewens sampling formula with two parameters @math , found by Pitman (1992), then the moment formula yields the Cauchy-Stieltjes transform of an @math mean. The analysis of these random means includes the characterization of @math -means, known as Dirichlet means, due to Von Neumann (1941), Watson (1956) and Cifarelli and Regazzini (1990) and generalizations of L 'evy's arcsine law for the time spent positive by a Brownian motion, due to Darling (1949) Lamperti (1958) and Barlow, Pitman and Yor (1989). In this paper, we propose a framework to study a general class of strategic behavior in voting, which we call vote operations. We prove the following theorem: if we fix the number of alternatives, generate @math votes i.i.d. according to a distribution @math , and let @math go to infinity, then for any @math , with probability at least @math , the minimum number of operations that are needed for the strategic individual to achieve her goal falls into one of the following four categories: (1) 0, (2) @math , (3) @math , and (4) @math . This theorem holds for any set of vote operations, any individual vote distribution @math , and any integer generalized scoring rule, which includes (but is not limited to) almost all commonly studied voting rules, e.g., approval voting, all positional scoring rules (including Borda, plurality, and veto), plurality with runoff, Bucklin, Copeland, maximin, STV, and ranked pairs. We also show that many well-studied types of strategic behavior fall under our framework, including (but not limited to) constructive destructive manipulation, bribery, and control by adding deleting votes, margin of victory, and minimum manipulation coalition size. Therefore, our main theorem naturally applies to these problems. A voter sits on each vertex of an infinite tree of degree @math , and has to decide between two alternative opinions. At each time step, each voter switches to the opinion of the majority of her neighbors. We analyze this majority process when opinions are initialized to independent and identically distributed random variables. In particular, we bound the threshold value of the initial bias such that the process converges to consensus. In order to prove an upper bound, we characterize the process of a single node in the large @math -limit. This approach is inspired by the theory of mean field spin-glass and can potentially be generalized to a wider class of models. We also derive a lower bound that is nontrivial for small, odd values of @math . Suppose that @math is a sequence of finite connected planar graphs, and in each @math a special vertex, called the root, is chosen randomly-uniformly. We introduce the notion of a distributional limit @math of such graphs. Assume that the vertex degrees of the vertices in @math are bounded, and the bound does not depend on @math . Then after passing to a subsequence, the limit exists, and is a random rooted graph @math . We prove that with probability one @math is recurrent. The proof involves the Circle Packing Theorem. The motivation for this work comes from the theory of random spherical triangulations.