A key property underlying the success of evolutionary algorithms (EAs) is their global search behavior, which allows the algorithms to jump' from a current state to other parts of the search space, thereby avoiding to get stuck in local optima. This property is obtained through a random choice of the radius at which offspring are sampled from previously evaluated solutions. It is well known that, thanks to this global search behavior, the probability that an EA using standard bit mutation finds a global optimum of an arbitrary function @math tends to one as the number of function evaluations grows. This advantage over heuristics using a fixed search radius, however, comes at the cost of using non-optimal step sizes also in those regimes in which the optimal rate is stable for a long time. This downside results in significant performance losses for many standard benchmark problems. We introduce in this work a simple way to interpolate between the random global search of EAs and their deterministic counterparts which sample from a fixed radius only. To this end, we introduce , in which the binomial choice of the search radius is replaced by a normal distribution. Normalized standard bit mutation allows a straightforward way to control its variance, and hence the degree of randomness involved. We experiment with a self-adjusting choice of this variance, and demonstrate its effectiveness for the two classic benchmark problems LeadingOnes and OneMax. Our work thereby also touches a largely ignored question in discrete evolutionary computation: multi-dimensional parameter control.
As reasoned above, normalized standard bit mutation offers an elegant way to interpolate between deterministic mutation strengths and regular standard bit mutation, thus showing that Randomized Local Search (RLS) variants with their deterministic search radii and the (1+1) EA with mutation rate @math are essentially just different instantiations of the same meta-algorithm. Similar results also extend to population-based @math EAs. Note that normalized standard bit mutation also allows other degrees of randomization, thereby offering a wide range for further experimentation. In this context we note that for the special case of standard RLS (i.e., the greedy (1+1) hill climber that flips in each iteration exactly one uniformly chosen bit) a similar meta-model allowing to interpolate between the (1+1) EA and RLS is the (1+1) EA @math introduced in @cite_2 @cite_5 . This model, however, is much less flexible, and does not allow, for example, deterministic search radii greater than one.
@cite_5 @cite_2
2888223476 2012697403
We study the well-known black-box optimisation algorithm (1+1)-EA on a novel type of noise model. In our noise model, the fitness function is linear with positive weights, but the absolute values of the weights may fluctuate in each round. Thus in every state, the fitness function indicates that one-bits are preferred over zero-bits. In particular, hillclimbing heuristics should be able to find the optimum fast. We show that the (1+1)-EA indeed finds the optimum in time @math if the mutation parameter is @math for a constant @math However, we also show that for @math the (1+1)-EA needs superpolynomial time to find the optimum. Thus the choice of mutation parameter is critical even for optimisation tasks in which there is a clear path to the the optimum. A similar threshold phenomenon has recently been shown for noise-free monotone fitness functions. The ( @math 1+ź) EA with mutation probability c n, where @math c>0 is an arbitrary constant, is studied for the classical OneMax function. Its expected optimization time is analyzed exactly (up to lower order terms) as a function of c and @math ź. It turns out that 1 n is the only optimal mutation probability if @math ź=o(lnnlnlnn lnlnlnn), which is the cut-off point for linear speed-up. However, if @math ź is above this cut-off point then the standard mutation probability 1 n is no longer the only optimal choice. Instead, the expected number of generations is (up to lower order terms) independent of c, irrespectively of it being less than 1 or greater. The theoretical results are obtained by a careful study of order statistics of the binomial distribution and variable drift theorems for upper and lower bounds. Experimental supplements shed light on the optimal mutation probability for small problem sizes. Extending previous analyses on function classes like linear functions, we analyze how the simple 1+1 evolutionary algorithm optimizes pseudo-Boolean functions that are strictly monotonic. These functions have the property that whenever only 0-bits are changed to 1, then the objective value strictly increases. Contrary to what one would expect, not all of these functions are easy to optimize. The choice of the constant c in the mutation probability pn=c n can make a decisive difference. We show that if c iterations. For c=1, we can still prove an upper bound of On3 2. However, for , we present a strictly monotonic function such that the 1+1 EA with overwhelming probability needs iterations to find the optimum. This is the first time that we observe that a constant factor change of the mutation probability changes the runtime by more than a constant factor. Extending previous analyses on function classes like linear functions, we analyze how the simple (1+1) evolutionary algorithm optimizes pseudo-Boolean functions that are strictly monotone. Contrary to what one would expect, not all of these functions are easy to optimize. The choice of the constant c in the mutation probability p(n) = c n can make a decisive difference. We show that if c > 1, then the (1+1) EA finds the optimum of every such function in Θ(n log n) iterations. For c = 1, we can still prove an upper bound of O(n3 2). However, for c > 33, we present a strictly monotone function such that the (1+1) EA with overwhelming probability does not find the optimum within 2Ω(n) iterations. This is the first time that we observe that a constant factor change of the mutation probability changes the run-time by more than constant factors. Evolving gradient-learning artificial neural networks (ANNs) using an evolutionary algorithm (EA) is a popular approach to address the local optima and design problems of ANN. The typical approach is to combine the strength of backpropagation (BP) in weight learning and EA's capability of searching the architecture space. However, the BP's "gradient descent" approach requires a highly computer-intensive operation that relatively restricts the search coverage of EA by compelling it to use a small population size. To address this problem, we utilized mutation-based genetic neural network (MGNN) to replace BP by using the mutation strategy of local adaptation of evolutionary programming (EP) to effect weight learning. The MGNN's mutation enables the network to dynamically evolve its structure and adapt its weights at the same time. Moreover, MGNN's EP-based encoding scheme allows for a flexible and less restricted formulation of the fitness function and makes fitness computation fast and efficient. This makes it feasible to use larger population sizes and allows MGNN to have a relatively wide search coverage of the architecture space. MGNN implements a stopping criterion where overfitness occurrences are monitored through "sliding-windows" to avoid premature learning and overlearning. Statistical analysis of its performance to some well-known classification problems demonstrate its good generalization capability. It also reveals that locally adapting or scheduling the strategy parameters embedded in each individual network may provide a proper balance between the local and global searching capabilities of MGNN. In practical applications evaluating a fitness function is frequently subject to noise, i. e., the “true fitness” is disturbed by some random variations. Evolutionary algorithms (EAs) are often successfully applied to noisy problems, where they have turned out to be particularly robust. Theoretical results on the behavior of EAs for noisy functions are comparatively very rare, especially for discrete search spaces. Here we present an analysis of the (1 + 1) EA for a noisy variant of OneMax and compute the maximal noise strength allowing the (1 + 1) EA a polynomial runtime asymptotically exactly. The methods used in the proofs are presented in a general form with clearly stated conditions in order to simplify further applications. Evolutionary algorithms are randomized search heuristics, which are applied to problems whose structure is not well understood, as well as to problems in combinatorial optimization. They have successfully been applied to different kinds of arc routing problems. To start the analysis of evolutionary algorithms with respect to the expected optimization time on these problems, we consider the Eulerian cycle problem. We show that a variant of the well-known (1+1) EA working on the important encoding of permutations is able to find an Eulerian tour of an Eulerian graph in expected polynomial time. Altering the operator used for mutation in the considered algorithms, the expected optimization time changes from polynomial to exponential. Beginning with the early days of the genetic algorithm and the schema theorem it has often been argued that the crossover operator is the more important genetic operator. The early Royal Road functions were put forth as an example where crossover would excel, yet mutation based EAs were subsequently shown to experimentally outperform GAs with crossover on these functions. Recently several new Royal Roads have been introduced and proved to require expected polynomial time for GAs with crossover, while needing exponential time to optimize for mutation-only EAs. This paper does the converse, showing proofs that GAs with crossover require exponential optimization time on new Ignoble Trail functions while mutation based EAs optimize them efficiently. A multi-population based approach is proposed to realize the adapted ensemble of multiple strategies of differential evolution.The control parameters of each mutation strategy are adapted independently.Extensive experiments are conducted to test the performance of multi-population ensemble DE (MPEDE). Differential evolution (DE) is among the most efficient evolutionary algorithms (EAs) for global optimization and now widely applied to solve diverse real-world applications. As the most appropriate configuration of DE to efficiently solve different optimization problems can be significantly different, an appropriate combination of multiple strategies into one DE variant attracts increasing attention recently. In this study, we propose a multi-population based approach to realize an ensemble of multiple strategies, thereby resulting in a new DE variant named multi-population ensemble DE (MPEDE) which simultaneously consists of three mutation strategies, i.e., "current-to-pbest 1" and "current-to-rand 1" and "rand 1". There are three equally sized smaller indicator subpopulations and one much larger reward subpopulation. Each constituent mutation strategy has one indicator subpopulation. After every certain number of generations, the current best performing mutation strategy will be determined according to the ratios between fitness improvements and consumed function evaluations. Then the reward subpopulation will be allocated to the determined best performing mutation strategy dynamically. As a result, better mutation strategies obtain more computational resources in an adaptive manner during the evolution. The control parameters of each mutation strategy are adapted independently as well. Extensive experiments on the suit of CEC 2005 benchmark functions and comprehensive comparisons with several other efficient DE variants show the competitive performance of the proposed MPEDE (Matlab codes of MPEDE are available from http: guohuawunudt.gotoip2.com publications.html). Evolutionary and genetic algorithms (EAs and GAs) are quite successful randomized function optimizers. This success is mainly based on the interaction of different operators like selection, mutation, and crossover. Since this interaction is still not well understood, one is interested in the analysis of the single operators. Jansen and Wegener [Proceedings of GECCO'2001, 2001, pp. 375-382] have described so-called real royal road functions where simple steady-state GAs have a polynomial expected optimization time while the success probability of mutation-based EAs is exponentially small even after an exponential number of steps. This success of the GA is based on the crossover operator and a population whose size is moderately increasing with the dimension of the search space. Here new real royal road functions are presented where crossover leads to a small optimization time, although the GA works with the smallest possible population size--namely 2. Due to experimental evidence it is incontestable that crossover is essential for some fitness functions. However, theoretical results without assumptions are difficult. So-called real royal road functions are known where crossover is proved to be essential, i.e., mutation-based algorithms have an exponential expected runtime while the expected runtime of a genetic algorithm is polynomially bounded. However, these functions are artificial and have been designed in such a way that crossover is essential only at the very end (or at other well-specified points) of the optimization process.Here, a more natural fitness function based on a generalized Ising model is presented where crossover is essential throughout the whole optimization process. Mutation-based algorithms such as (μ+λ) EAs with constant population size are proved to have an exponential expected runtime while the expected runtime of a simple genetic algorithm with population size 2 and fitness sharing is polynomially bounded. Parameterized runtime analysis seeks to understand the influence of problem structure on algorithmic runtime. In this paper, we contribute to the theoretical understanding of evolutionary algorithms and carry out a parameterized analysis of evolutionary algorithms for the Euclidean traveling salesperson problem (Euclidean TSP). We investigate the structural properties in TSP instances that influence the optimization process of evolutionary algorithms and use this information to bound the runtime of simple evolutionary algorithms. Our analysis studies the runtime in dependence of the number of inner points @math and shows that @math evolutionary algorithms solve the Euclidean TSP in expected time @math where @math is a function of the minimum angle @math between any three points. Finally, our analysis provides insights into designing a mutation operator that improves the upper bound on expected runtime. We show that a mixed mutation strategy that incorporates both 2-opt moves and permutation jumps results in an upper bound of @math for the @math EA. Over the past 30 years numerous algorithms have been designed for symmetry breaking problems in the LOCAL model, such as maximal matching, MIS, vertex coloring, and edge-coloring. For most problems the best randomized algorithm is at least exponentially faster than the best deterministic algorithm. In this paper we prove that these exponential gaps are necessary and establish connections between the deterministic and randomized complexities in the LOCAL model. Each result has a very compelling take-away message: 1. Fast @math -coloring of trees requires random bits: Building on the recent lower bounds of , we prove that the randomized complexity of @math -coloring a tree with maximum degree @math is @math , whereas its deterministic complexity is @math for any @math . This also establishes a large separation between the deterministic complexity of @math -coloring and @math -coloring trees. 2. Randomized lower bounds imply deterministic lower bounds: We prove that any deterministic algorithm for a natural class of problems that runs in @math rounds can be transformed to run in @math rounds. If the transformed algorithm violates a lower bound (even allowing randomization), then one can conclude that the problem requires @math time deterministically. 3. Deterministic lower bounds imply randomized lower bounds: We prove that the randomized complexity of any natural problem on instances of size @math is at least its deterministic complexity on instances of size @math . This shows that a deterministic @math lower bound for any problem implies a randomized @math lower bound. It also illustrates that the graph shattering technique is absolutely essential to the LOCAL model. Unique input–output (UIO) sequences have important applications in conformance testing of finite state machines (FSMs). Previous experimental and theoretical research has shown that evolutionary algorithms (EAs) can compute UIOs efficiently on many FSM instance classes, but fail on others. However, it has been unclear how and to what degree EA parameter settings influence the runtime on the UIO problem. This paper investigates the choice of acceptance criterion in the (1 + 1) EA and the use of crossover in the @math Steady State Genetic Algorithm. It is rigorously proved that changing these parameters can reduce the runtime from exponential to polynomial for some instance classes of the UIO problem. We reinvestigate a fundamental question: How effective is crossover in genetic algorithms in combining building blocks of good solutions? Although this has been discussed controversially for decades, we are still lacking a rigorous and intuitive answer. We provide such answers for royal road functions and OneMax, where every bit is a building block. For the latter, we show that using crossover makes every + i¾ genetic algorithm at least twice as fast as the fastest evolutionary algorithm using only standard bit mutation, up to small-order terms and for moderatei¾ andi¾ . Crossover is beneficial because it can capitalize on mutations that have both beneficial and disruptive effects on building blocks: crossover is able to repair the disruptive effects of mutation in later generations. Compared to mutation-based evolutionary algorithms, this makes multibit mutations more useful. Introducing crossover changes the optimal mutation rate on OneMax from to . This holds both for uniform crossover and k-point crossover. Experiments and statistical tests confirm that our findings apply to a broad class of building block functions. The Moran process, as studied by Lieberman, Hauert, and Nowak, is a randomised algorithm modelling the spread of genetic mutations in populations. The algorithm runs on an underlying graph where individuals correspond to vertices. Initially, one vertex (chosen uniformly at random) possesses a mutation, with fitness r > 1. All other individuals have fitness 1. During each step of the algorithm, an individual is chosen with probability proportional to its fitness, and its state (mutant or nonmutant) is passed on to an out-neighbour which is chosen uniformly at random. If the underlying graph is strongly connected, then the algorithm will eventually reach fixation, in which all individuals are mutants, or extinction, in which no individuals are mutants. An infinite family of directed graphs is said to be strongly amplifying if, for every r > 1, the extinction probability tends to 0 as the number of vertices increases. A formal definition is provided in the article. Strong amplification is a rather surprising property—it means that in such graphs, the fixation probability of a uniformly placed initial mutant tends to 1 even though the initial mutant only has a fixed selective advantage of r > 1 (independently of n). The name “strongly amplifying” comes from the fact that this selective advantage is “amplified.” Strong amplifiers have received quite a bit of attention, and proposed two potentially strongly amplifying families—superstars and metafunnels. Heuristic arguments have been published, arguing that there are infinite families of superstars that are strongly amplifying. The same has been claimed for metafunnels. In this article, we give the first rigorous proof that there is an infinite family of directed graphs that is strongly amplifying. We call the graphs in the family “megastars.” When the algorithm is run on an n-vertex graph in this family, starting with a uniformly chosen mutant, the extinction probability is roughly n − 1 2 (up to logarithmic factors). We prove that all infinite families of superstars and metafunnels have larger extinction probabilities (as a function of n). Finally, we prove that our analysis of megastars is fairly tight—there is no infinite family of megastars such that the Moran algorithm gives a smaller extinction probability (up to logarithmic factors). Also, we provide a counterexample which clarifies the literature concerning the isothermal theorem of Abstract The optimal bucket order problem consists in obtaining a complete consensus ranking (ties are allowed) from a matrix of preferences (possibly obtained from a database of rankings). In this paper, we tackle this problem by using ( 1 + λ ) evolution strategies. We designed specific mutation operators which are able to modify the inner structure of the buckets, which introduces more diversity into the search process. We also study different initialization methods and strategies for the generation of the population of descendants. The proposed evolution strategies are tested using a benchmark of 52 databases and compared with the current state-of-the-art algorithm LIA G M P 2 . We carry out a standard machine learning statistical analysis procedure to identify a subset of outstanding configurations of the proposed evolution strategies. The study shows that the best evolution strategy improves upon the accuracy obtained by the standard greedy method (BPA) by 35 , and that of LIA G M P 2 by 12.5 . Mutations are required for adaptation, yet most mutations with phenotypic effects are deleterious. As a consequence, the mutation rate that maximizes adaptation will be some intermediate value. This abstract summarizes a previous publication in which we used Avida, a well-studied artificial life platform, to investigate the ability of natural selection to adjust and optimize mutation rates. Our initial experiments occurred in a previously studied environment with a complex fitness landscape ( Nature, 423, 2003) where Avidians were rewarded for performing any of nine logic tasks. We assessed the optimal mutation rate by empirically determining which unchanging mutation rate produced the highest rate of adaptation. Then, we allowed mutation rates to evolve and we evaluated their proximity to the optimum. Although we chose conditions favorable for mutation rate optimization (asexual organisms not yet adapted to a new environment), the evolved rates were invariably far below the optimum across a wide range of experimental parameter settings (Fig. 1). We hypothesized that the reason mutation rates evolved to be suboptimal was the ruggedness of fitness landscapes. To test this hypothesis, we created a simplified 'counting ones' landscape without any fitness valleys and found that, in such conditions, populations evolved near-optimal mutation rates (Fig. 2, top row). In contrast, once moderate fitness valleys were added to this simple landscape, the ability of evolving populations to find the optimal mutation rate was lost (Fig. 2, bottom two rows). Additional experiments revealed that lowering the rate at which mutation rates evolved did not preclude the evolution of suboptimal mutation rates (see original manuscript). We conclude that rugged fitness landscapes can prevent the evolution of mutation rates that are optimal for long-term adaptation because of the short-term costs of traversing fitness valleys. This finding has important implications for evolutionary research in both biological and computational realms. We consider the classic Moran process modeling the spread of genetic mutations, as extended to structured populations by (Nature, 2005). In this process, individuals are the vertices of a connected graph G. Initially, there is a single mutant vertex, chosen uniformly at random. In each step, a random vertex is selected for reproduction with a probability proportional to its fitness: mutants have fitness r > 1, while non-mutants have fitness 1. The vertex chosen to reproduce places a copy of itself to a uniformly random neighbor in G, replacing the individual that was there. The process ends when the mutation either reaches fixation (i.e., all vertices are mutants), or gets extinct. The principal quantity of interest is the probability with which each of the two outcomes occurs. A problem that has received significant attention recently concerns the existence of families of graphs, called strong amplifiers of selection, for which the fixation probability tends to 1 as the order n of the graph increases, and the existence of strong suppressors of selection, for which this probability tends to 0. For the case of directed graphs, it is known that both strong amplifiers and suppressors exist. For the case of undirected graphs, however, the problem has remained open, and the general belief has been that neither strong amplifiers nor suppressors exist. In this paper we disprove this belief, by providing the first examples of such graphs. The strong amplifier we present has fixation probability 1 − ˜ O(n^ −1 3), and the strong suppressor has fixation probability ˜O(n^−1 4). Both graph constructions are surprisingly simple. We also prove a general upper bound of 1 − ˜Ω(n^−1 3) on the fixation probability of any undirected graph. Hence, our strong amplifier is existentially optimal. Evolutionary algorithms (EAs) are often well-suited for optimization problems involving several, often conflicting objectives. Since 1985, various evolutionary approaches to multiobjective optimization have been developed that are capable of searching for multiple solutions concurrently in a single run. However, the few comparative studies of different methods presented up to now remain mostly qualitative and are often restricted to a few approaches. In this paper, four multiobjective EAs are compared quantitatively where an extended 0 1 knapsack problem is taken as a basis. Furthermore, we introduce a new evolutionary approach to multicriteria optimization, the strength Pareto EA (SPEA), that combines several features of previous multiobjective EAs in a unique manner. It is characterized by (a) storing nondominated solutions externally in a second, continuously updated population, (b) evaluating an individual's fitness dependent on the number of external nondominated points that dominate it, (c) preserving population diversity using the Pareto dominance relationship, and (d) incorporating a clustering procedure in order to reduce the nondominated set without destroying its characteristics. The proof-of-principle results obtained on two artificial problems as well as a larger problem, the synthesis of a digital hardware-software multiprocessor system, suggest that SPEA can be very effective in sampling from along the entire Pareto-optimal front and distributing the generated solutions over the tradeoff surface. Moreover, SPEA clearly outperforms the other four multiobjective EAs on the 0 1 knapsack problem.