An approach to the acceleration of parametric weak classifier boosting is proposed. Weak classifier is called parametric if it has fixed number of parameters and, so, can be represented as a point into multidimensional space. Genetic algorithm is used instead of exhaustive search to learn parameters of such classifier. Proposed approach also takes cases when effective algorithm for learning some of the classifier parameters exists into account. Experiments confirm that such an approach can dramatically decrease classifier training time while keeping both training and test errors small.
Usage of genetic algorithm for weak learner acceleration was already proposed in several works. For example, in @cite_0 genetic weak learner with special crossover and mutation operators was used to learn classifier based on extended haar feature set. In @cite_7 genetic algorithm was used to select a few thousand weak classifiers with smallest error on unweighed training set before boosting process starts. Then exhaustive search over selected classifiers was performed on each boosting iteration to select the one with minimal weighed loss. In @cite_3 boosting procedure was completly integrated with genetic algorithm. Few classifiers were selected on each boosting iteration from solution population and added to the strong classifier. That selected classifiers were then used to produce new population members by applying genetic operators. Then, in @cite_4 authors used for weak learner some special evolutionary algorithm they've called Evolutionary Hill-Climbing . Crossover operator was not used in it. Instead, @math different mutations were applied to every population member on each algorithm iteration. Result of each mutation was rejected when it did not improve fitness function value.
@cite_0 @cite_3 @cite_4 @cite_7
2118399788 2153485504 2026182474
Mutation and crossover are the main search operators of different variants of evolutionary algorithms. Despite the many discussions on the importance of crossover nobody has proved rigorously for some explicitly defined fitness functions f"n: 0,1 ^n->R that a genetic algorithm with crossover can optimize f"n in expected polynomial time while all evolution strategies based only on mutation (and selection) need expected exponential time. Here such functions and proofs are presented for a genetic algorithm without any idealization. For some functions one-point crossover is appropriate while for others uniform crossover is the right choice. We propose a new approach for designing classifiers for a c-class (c spl ges 2) problem using genetic programming (GP). The proposed approach takes an integrated view of all classes when the GP evolves. A multitree representation of chromosomes is used. In this context, we propose a modified crossover operation and a new mutation operation that reduces the destructive nature of conventional genetic operations. We use a new concept of unfitness of a tree to select trees for genetic operations. This gives more opportunity to unfit trees to become fit. A new concept of OR-ing chromosomes in the terminal population is introduced, which enables us to get a classifier with better performance. Finally, a weight-based scheme and some heuristic rules characterizing typical ambiguous situations are used for conflict resolution. The classifier is capable of saying "don't know" when faced with unfamiliar examples. The effectiveness of our scheme is demonstrated on several real data sets. Evolutionary and genetic algorithms (EAs and GAs) are quite successful randomized function optimizers. This success is mainly based on the interaction of different operators like selection, mutation, and crossover. Since this interaction is still not well understood, one is interested in the analysis of the single operators. Jansen and Wegener [Proceedings of GECCO'2001, 2001, pp. 375-382] have described so-called real royal road functions where simple steady-state GAs have a polynomial expected optimization time while the success probability of mutation-based EAs is exponentially small even after an exponential number of steps. This success of the GA is based on the crossover operator and a population whose size is moderately increasing with the dimension of the search space. Here new real royal road functions are presented where crossover leads to a small optimization time, although the GA works with the smallest possible population size--namely 2. This paper addresses a Markov chain analysis of genetic algorithms (GAs), in particular for a variety called a modified elitist strategy. The modified elitist strategy generates the current population of M individuals by reserving the individual with the highest fitness value from the previous generation and generating M-1 individuals through a generation change. The author's analysis is based on a Markov chain: by assuming a simple GA in which the genetic operation in the generation changes is restricted to selection, crossover, and mutation, and by evaluating the eigenvalues of the transition matrix of the Markov chain, the convergence rate of the GAs is computed in terms of a mutation probability spl mu . In this way, the authors show the probability that the population includes the individual with the highest fitness value is lower-bounded by 1-O(| spl lambda *| sup n ), | spl lambda *| > Abstract Genetic algorithms methods utilize the same optimization procedures as natural genetic evolution, in which a population is gradually improved by selection. We have developed a genetic algorithm search procedure suitable for use in protein folding simulations. A population of conformations of the polypeptide chain is maintained, and conformations are changed by mutation, in the form of conventional Monte Carlo steps, and crossovers in which parts of the polypeptide chain are interchanged between conformations. For folding on a simple two-dimensional lattice it is found that the genetic algorithm is dramatically superior to conventional Monte Carlo methods. We propose a hybrid algorithm for finding a set of nondominated solutions of a multi objective optimization problem. In the proposed algorithm, a local search procedure is applied to each solution (i.e., each individual) generated by genetic operations. Our algorithm uses a weighted sum of multiple objectives as a fitness function. The fitness function is utilized when a pair of parent solutions are selected for generating a new solution by crossover and mutation operations. A local search procedure is applied to the new solution to maximize its fitness value. One characteristic feature of our algorithm is to randomly specify weight values whenever a pair of parent solutions are selected. That is, each selection (i.e., the selection of two parent solutions) is performed by a different weight vector. Another characteristic feature of our algorithm is not to examine all neighborhood solutions of a current solution in the local search procedure. Only a small number of neighborhood solutions are examined to prevent the local search procedure from spending almost all available computation time in our algorithm. High performance of our algorithm is demonstrated by applying it to multi objective flowshop scheduling problems. There is a great interest in the Genetic Programming (GP) community to develop semantic genetic operators. Most recent approaches includes the genetic programming framework for symbolic regression called Error Space Alignment GP, the geometric semantic operators, and our previous work the semantic crossover based on the partial derivative error. To the best of our knowledge, there has not been a semantic genetic operator similar to the point mutation. In this contribution, we start filling this gap by proposing a semantic point mutation based on the derivative of the error. This novel operator complements our previous semantic crossover and, as the results show, there is an improvement in performance when this novel operator is used, and, furthermore, the best performance in our setting is the system that uses the semantic crossover and the semantic point mutation. This paper presents algorithm for optimal reconfiguration of distribution networks using hybrid heuristic genetic algorithm. Improvements introduced in this approach make it suitable for real-life networks with realistic degree of complexity and network size. The algorithm introduces several improvements related to the generation of initial set of possible solutions as well as crossover and mutation steps in genetic algorithm. Since the genetic algorithms are often used in distribution network reconfiguration problem, its application is well known, but most of the approaches have very poor effectiveness due to high level of individuals' rejections not-fulfilling radial network constraints requirements and poor convergence rate. One part of these problems is related to ineffective creation of initial population individuals. The other part of the problem in similar approaches is related to inefficient operators implemented in crossover and mutation process over created set of population individuals. The hybrid heuristic-genetic approach presented in this paper provides significant improvements in these areas. The presented algorithm can be used to find optimal radial distribution network topology with minimum network losses or with optimally balanced network loading. The algorithm is rested on a real size network of city of Dubrovnik to identify the optimal network topology after the interpolation (connection) of a new supply point. This paper presents an online feature selection algorithm using genetic programming (GP). The proposed GP methodology simultaneously selects a good subset of features and constructs a classifier using the selected features. For a c-class problem, it provides a classifier having c trees. In this context, we introduce two new crossover operations to suit the feature selection process. As a byproduct, our algorithm produces a feature ranking scheme. We tested our method on several data sets having dimensions varying from 4 to 7129. We compared the performance of our method with results available in the literature and found that the proposed method produces consistently good results. To demonstrate the robustness of the scheme, we studied its effectiveness on data sets with known (synthetically added) redundant bad features. Training a cascade-based face detector using boosting and Haar features is computationally expensive, often requiring weeks on single CPU machines. The bottleneck is at training and selecting Haar features for a single weak classifier, currently in minutes. Traditional techniques for training a weak classifier usually run in 0(NT log N), with N examples (approximately 10,000), and T features (approximately 40,000). We present a method to train a weak classifier in time 0(Nd2 + T), where d is the number of pixels of the probed image sub-window (usually from 350 to 500), by using only the statistics of the weighted input data. Experimental results revealed a significantly reduced training time of a weak classifier to the order of seconds. In particular, this method suffers very minimal immerse in training time with very large increases in members of Haar features, enjoying a significant gain in accuracy, even with reduced training time. Due to experimental evidence it is incontestable that crossover is essential for some fitness functions. However, theoretical results without assumptions are difficult. So-called real royal road functions are known where crossover is proved to be essential, i.e., mutation-based algorithms have an exponential expected runtime while the expected runtime of a genetic algorithm is polynomially bounded. However, these functions are artificial and have been designed in such a way that crossover is essential only at the very end (or at other well-specified points) of the optimization process.Here, a more natural fitness function based on a generalized Ising model is presented where crossover is essential throughout the whole optimization process. Mutation-based algorithms such as (μ+λ) EAs with constant population size are proved to have an exponential expected runtime while the expected runtime of a simple genetic algorithm with population size 2 and fitness sharing is polynomially bounded. Boosting-based object detection has received significant attention recently. In this paper, we propose totally corrective asymmetric boosting algorithms for real-time object detection. Our algorithms differ from Viola and Jones' detection framework in two ways. Firstly, our boosting algorithms explicitly optimize asymmetric loss of objectives, while AdaBoost used by Viola and Jones optimizes a symmetric loss. Secondly, by carefully deriving the Lagrange duals of the optimization problems, we design more efficient boosting in that the coefficients of the selected weak classifiers are updated in a totally corrective fashion, in contrast to the stagewise optimization commonly used by most boosting algorithms. Column generation is employed to solve the proposed optimization problems. Unlike conventional boosting, the proposed boosting algorithms are able to de-select those irrelevant weak classifiers in the ensemble while training a classification cascade. This results in improved detection performance as well as fewer weak classifiers in the learned strong classifier. Compared with AsymBoost of Viola and Jones, our proposed asymmetric boosting is nonheuristic and the training procedure is much simpler. Experiments on face and pedestrian detection demonstrate that our methods have superior detection performance than some of the state-of-the-art object detectors. We reinvestigate a fundamental question: How effective is crossover in genetic algorithms in combining building blocks of good solutions? Although this has been discussed controversially for decades, we are still lacking a rigorous and intuitive answer. We provide such answers for royal road functions and OneMax, where every bit is a building block. For the latter, we show that using crossover makes every + i¾ genetic algorithm at least twice as fast as the fastest evolutionary algorithm using only standard bit mutation, up to small-order terms and for moderatei¾ andi¾ . Crossover is beneficial because it can capitalize on mutations that have both beneficial and disruptive effects on building blocks: crossover is able to repair the disruptive effects of mutation in later generations. Compared to mutation-based evolutionary algorithms, this makes multibit mutations more useful. Introducing crossover changes the optimal mutation rate on OneMax from to . This holds both for uniform crossover and k-point crossover. Experiments and statistical tests confirm that our findings apply to a broad class of building block functions. Understanding the impact of crossover on performance is a major problem in the theory of genetic algorithms (GAs). We present new insight on working principles of crossover by analyzing the performance of crossover-based GAs on the simple functions OneMax and Jump. First, we assess the potential speedup by crossover when combined with a fitness-invariant bit shuffling operator that simulates a lineage of independent evolution on a function of unitation. Theoretical and empirical results show drastic speedups for both functions. Second, we consider a simple GA without shuffling and investigate the interplay of mutation and crossover on Jump. If the crossover probability is small, subsequent mutations create sufficient diversity, even for very small populations. Contrarily, with high crossover probabilities crossover tends to lose diversity more quickly than mutation can create it. This has a drastic impact on the performance on Jump. We complement our theoretical findings by Monte Carlo simulations on the population diversity. This paper presents a new algorithm based on integrating genetic algorithms, tabu search and simulated annealing methods to solve the unit commitment problem. The core of the proposed algorithm is based on genetic algorithms. Tabu search is used to generate new population members in the reproduction phase of the genetic algorithm. A simulated annealing method is used to accelerate the convergence of the genetic algorithm by applying the simulated annealing test for all the population members. A new implementation of the genetic algorithm is introduced. The genetic algorithm solution is coded as a mix between binary and decimal representation. The fitness function is constructed from the total operating cost of the generating units without penalty terms. In the tabu search part of the proposed algorithm, a simple short-term memory procedure is used to counter the danger of entrapment at a local optimum, and the premature convergence of the genetic algorithm. A simple cooling schedule has been implemented to apply the simulated annealing test in the algorithm. Numerical results showed the superiority of the solutions obtained compared to genetic algorithms, tabu search and simulated annealing methods, and to two exact algorithms. A parallel version of a new hybrid genetic algorithm for the vehicle routing problem with time windows is presented. The route-directed hybrid genetic approach is based upon the simultaneous evolution of two populations of solutions focusing on separate objectives subject to temporal constraint relaxation. While the first population evolves individuals to minimize total traveled distance the second aims at minimizing temporal constraint violation to generate a feasible solution. Genetic operators have been designed to capture key concepts from successful routing techniques to further enhance search diversification and intensification. A master–slave message-passing paradigm characterizes the parallel procedure. The master component controls the execution of the algorithm, coordinates genetic operations and handles parent selection while the slave elements concurrently execute reproduction and mutation operators. Providing additional speed-up, the parallel algorithm further expands on its sequential counterpart, matching or even improving solution quality. Computational results show the proposed technique to be very competitive with the best-known heuristic routing procedures providing some new best-known solutions. This paper shows a theoretical property on the Markov chain of genetic algorithms: the stationary distribution focuses on the uniform population with the optimal solution as mutation and crossover probabilities go to zero and some selective pressure defined in this paper goes to infinity. Moreover, as a result, a sufficient condition for ergodicity is derived when a simulated annealing-like strategy is considered. Additionally, the uniform crossover counterpart of the Vose-Liepins formula is derived using the Markov chain model. Beginning with the early days of the genetic algorithm and the schema theorem it has often been argued that the crossover operator is the more important genetic operator. The early Royal Road functions were put forth as an example where crossover would excel, yet mutation based EAs were subsequently shown to experimentally outperform GAs with crossover on these functions. Recently several new Royal Roads have been introduced and proved to require expected polynomial time for GAs with crossover, while needing exponential time to optimize for mutation-only EAs. This paper does the converse, showing proofs that GAs with crossover require exponential optimization time on new Ignoble Trail functions while mutation based EAs optimize them efficiently. A novel learning procedure, called FloatBoost, is proposed for learning a boosted classifier for achieving the minimum error rate. FloatBoost learning uses a backtrack mechanism after each iteration of AdaBoost learning to minimize the error rate directly, rather than minimizing an exponential function of the margin as in the traditional AdaBoost algorithms. A second contribution of the paper is a novel statistical model for learning best weak classifiers using a stagewise approximation of the posterior probability. These novel techniques lead to a classifier which requires fewer weak classifiers than AdaBoost yet achieves lower error rates in both training and testing, as demonstrated by extensive experiments. Applied to face detection, the FloatBoost learning method, together with a proposed detector pyramid architecture, leads to the first real-time multiview face detection system reported. A training process for facial expression recognition is usually performed sequentially in three individual stages: feature learning, feature selection, and classifier construction. Extensive empirical studies are needed to search for an optimal combination of feature representation, feature set, and classifier to achieve good recognition performance. This paper presents a novel Boosted Deep Belief Network (BDBN) for performing the three training stages iteratively in a unified loopy framework. Through the proposed BDBN framework, a set of features, which is effective to characterize expression-related facial appearance shape changes, can be learned and selected to form a boosted strong classifier in a statistical way. As learning continues, the strong classifier is improved iteratively and more importantly, the discriminative capabilities of selected features are strengthened as well according to their relative importance to the strong classifier via a joint fine-tune process in the BDBN framework. Extensive experiments on two public databases showed that the BDBN framework yielded dramatic improvements in facial expression analysis.