Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) C,C,?,? Needs to be more explored: <s> This chapter introduces and overviews an emerging methodology in search and optimisation. One of the key aims of these new approaches, which have been termed hyperheuristics, is to raise the level of generality at which optimisation systems can operate. An objective is that hyper-heuristics will lead to more general systems that are able to handle a wide range of problem domains rather than current meta-heuristic technology which tends to be customised to a particular problem or a narrow class of problems. Hyper-heuristics are broadly concerned with intelligently choosing the right heuristic or algorithm in a given situation. Of course, a hyper-heuristic can be (often is) a (meta-)heuristic and it can operate on (meta-)heuristics. In a certain sense, a hyper-heuristic works at a higher level when compared with the typical application of meta-heuristics to optimisation problems, i.e., a hyper-heuristic could be thought of as a (meta)-heuristic which operates on lower level (meta-)heuristics. In this chapter we will introduce the idea and give a brief history of this emerging area. In addition, we will review some of the latest work to be published in the field. <s> BIB001 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) C,C,?,? Needs to be more explored: <s> The particle swarm optimizer (PSO) is a stochastic, population-based optimization technique that can be applied to a wide range of problems, including neural network training. This paper presents a variation on the traditional PSO algorithm, called the cooperative particle swarm optimizer, or CPSO, employing cooperative behavior to significantly improve the performance of the original algorithm. This is achieved by using multiple swarms to optimize different components of the solution vector cooperatively. Application of the new PSO algorithm on several benchmark optimization problems shows a marked improvement in performance over the traditional PSO. <s> BIB002 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) C,C,?,? Needs to be more explored: <s> Over the last decade, memetic algorithms (MAs) have relied on the use of a variety of different methods as the local improvement procedure. Some recent studies on the choice of local search method employed have shown that this choice significantly affects the efficiency of problem searches. Given the restricted theoretical knowledge available in this area and the limited progress made on mitigating the effects of incorrect local search method choice, we present strategies for MA control that decide, at runtime, which local method is chosen to locally improve the next chromosome. The use of multiple local methods during a MA search in the spirit of Lamarckian learning is here termed Meta-Lamarckian learning. Two adaptive strategies for Meta-Lamarckian learning are proposed in this paper. Experimental studies with Meta-Lamarckian learning strategies on continuous parametric benchmark problems are also presented. Further, the best strategy proposed is applied to a real-world aerodynamic wing design problem and encouraging results are obtained. It is shown that the proposed approaches aid designers working on complex engineering problems by reducing the probability of employing inappropriate local search methods in a MA, while at the same time, yielding robust and improved design search performance. <s> BIB003 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) C,C,?,? Needs to be more explored: <s> The combination of evolutionary algorithms with local search was named "memetic algorithms" (MAs) (Moscato, 1989). These methods are inspired by models of natural systems that combine the evolutionary adaptation of a population with individual learning within the lifetimes of its members. Additionally, MAs are inspired by Richard Dawkin's concept of a meme, which represents a unit of cultural evolution that can exhibit local refinement (Dawkins, 1976). In the case of MA's, "memes" refer to the strategies (e.g., local refinement, perturbation, or constructive methods, etc.) that are employed to improve individuals. In this paper, we review some works on the application of MAs to well-known combinatorial optimization problems, and place them in a framework defined by a general syntactic model. This model provides us with a classification scheme based on a computable index D, which facilitates algorithmic comparisons and suggests areas for future research. Also, by having an abstract model for this class of metaheuristics, it is possible to explore their design space and better understand their behavior from a theoretical standpoint. We illustrate the theoretical and practical relevance of this model and taxonomy for MAs in the context of a discussion of important design issues that must be addressed to produce effective and efficient MAs. <s> BIB004 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) C,C,?,? Needs to be more explored: <s> ABSTRACT This article presents a numerical procedure for automatically controlling desired features of a melt undergoing solidification by applying an external magnetic field whose time-varying intensity and spatial distribution are obtained by the use of a hybrid optimization algorithm. The intensities of the magnets along the boundaries of the container were discretized by using B-splines. The inverse problem is then formulated to find the magnetic boundary conditions (the coefficients of the B-splines) in such a way that the gradients of temperature along the gravity direction are minimized at each instant as the solidification front advances through a moving melt. For this task, a hybrid optimization code was used that automatically switches among the following six optimization modules; the Davidon-Fletcher-Powell (DFP) gradient method, a genetic algorithm (GA), the Nelder-Mead (NM) simplex method, quasi-Newton algorithm of Pshenichny-Danilin (LM), differential evolution (DE), and sequential quadratic... <s> BIB005 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) C,C,?,? Needs to be more explored: <s> Adaptation of parameters and operators represents one of the recent most important and promising areas of research in evolutionary computations; it is a form of designing self-configuring algorithms that acclimatize to suit the problem in hand. Here, our interests are on a recent breed of hybrid evolutionary algorithms typically known as adaptive memetic algorithms (MAs). One unique feature of adaptive MAs is the choice of local search methods or memes and recent studies have shown that this choice significantly affects the performances of problem searches. In this paper, we present a classification of memes adaptation in adaptive MAs on the basis of the mechanism used and the level of historical knowledge on the memes employed. Then the asymptotic convergence properties of the adaptive MAs considered are analyzed according to the classification. Subsequently, empirical studies on representatives of adaptive MAs for different type-level meme adaptations using continuous benchmark problems indicate that global-level adaptive MAs exhibit better search performances. Finally we conclude with some promising research directions in the area. <s> BIB006 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) C,C,?,? Needs to be more explored: <s> Hyperheuristics are single candidate solution based and simple to maintain mechanisms used in optimization. At each iteration, as a higher level of abstraction, a hyperheuristic chooses and applies one of the heuristics to a candidate solution. In this study, the performance contribution of hill climbing operators along with the mutational heuristics are analyzed in depth in four different hyperheuristic frameworks. Four different hill climbing operators and three mutational operators are used during the experiments. Various subsets of the heuristics are evaluated on fourteen well-known benchmark functions. <s> BIB007 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) C,C,?,? Needs to be more explored: <s> In this paper, we present a novel surrogate-assisted evolutionary optimization framework for solving computationally expensive problems. The proposed framework uses computationally cheap hierarchical surrogate models constructed through online learning to replace the exact computationally expensive objective functions during evolutionary search. At the first level, the framework employs a data-parallel Gaussian process based global surrogate model to filter the evolutionary algorithm (EA) population of promising individuals. Subsequently, these potential individuals undergo a memetic search in the form of Lamarckian learning at the second level. The Lamarckian evolution involves a trust-region enabled gradient-based search strategy that employs radial basis function local surrogate models to accelerate convergence. Numerical results are presented on a series of benchmark test functions and on an aerodynamic shape design problem. The results obtained suggest that the proposed optimization framework converges to good designs on a limited computational budget. Furthermore, it is shown that the new algorithm gives significant savings in computational cost when compared to the traditional evolutionary algorithm and other surrogate assisted optimization frameworks <s> BIB008 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) C,C,?,? Needs to be more explored: <s> Evolutionary algorithms (EAs) have been applied with success to many numerical and combinatorial optimization problems in recent years. However, they often lose their effectiveness and advantages when applied to large and complex problems, e.g., those with high dimensions. Although cooperative coevolution has been proposed as a promising framework for tackling high-dimensional optimization problems, only limited studies were reported by decomposing a high-dimensional problem into single variables (dimensions). Such methods of decomposition often failed to solve nonseparable problems, for which tight interactions exist among different decision variables. In this paper, we propose a new cooperative coevolution framework that is capable of optimizing large scale nonseparable problems. A random grouping scheme and adaptive weighting are introduced in problem decomposition and coevolution. Instead of conventional evolutionary algorithms, a novel differential evolution algorithm is adopted. Theoretical analysis is presented in this paper to show why and how the new framework can be effective for optimizing large nonseparable problems. Extensive computational studies are also carried out to evaluate the performance of newly proposed algorithm on a large number of benchmark functions with up to 1000 dimensions. The results show clearly that our framework and algorithm are effective as well as efficient for large scale evolutionary optimisation problems. We are unaware of any other evolutionary algorithms that can optimize 1000-dimension nonseparable problems as effectively and efficiently as we have done. <s> BIB009 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) C,C,?,? Needs to be more explored: <s> This paper attempts to address the question of scaling up Particle Swarm Optimization (PSO) algorithms to high dimensional optimization problems. We present a cooperative coevolving PSO (CCPSO) algorithm incorporating random grouping and adaptive weighting, two techniques that have been shown to be effective for handling high dimensional nonseparable problems. The proposed CCPSO algorithms out-performed a previously developed coevolving PSO algorithm on nonseparable functions of 30 dimensions. Furthermore, the scalability of the proposed algorithm to high dimensional nonseparable problems (of up to 1000 dimensions) is examined and compared with two existing coevolving Differential Evolution (DE) algorithms, and new insights are obtained. Our experimental results show the proposed CCPSO algorithms can perform reasonably well with only a small number of evaluations. The results also suggest that both the random grouping and adaptive weighting schemes are viable approaches that can be generalized to other evolutionary optimization methods. <s> BIB010 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) C,C,?,? Needs to be more explored: <s> Memetic algorithms (MAs) represent one of the recent growing areas in evolutionary algorithm (EA) research. The term MAs is now widely used as a synergy of evolutionary or any population-based approach with separate individual learning or local improvement procedures for problem search. Quite often, MAs are also referred to in the literature as Baldwinian EAs, Lamarckian EAs, cultural algorithms, or genetic local searches. In the last decade, MAs have been demonstrated to converge to high-quality solutions more efficiently than their conventional counterparts on a wide range of real-world problems. Despite the success and surge in interests on MAs, many of the successful MAs reported have been crafted to suit problems in very specific domains. Given the restricted theoretical knowledge available in the field of MAs and the limited progress made on formal MA frameworks, we present a novel probabilistic memetic framework that models MAs as a process involving the decision of embracing the separate actions of evolution or individual learning and analyzing the probability of each process in locating the global optimum. Further, the framework balances evolution and individual learning by governing the learning intensity of each individual according to the theoretical upper bound derived while the search progresses. Theoretical and empirical studies on representative benchmark problems commonly used in the literature are presented to demonstrate the characteristics and efficacies of the probabilistic memetic framework. Further, comparisons to recent state-of-the-art evolutionary algorithms, memetic algorithms, and hybrid evolutionary-local search demonstrate that the proposed framework yields robust and improved search performance. <s> BIB011
According to the analysis of existing DEPSOs and other representa-tive hybrids, the hybridization patterns C,P,?,? (especially, C,P,S,D ), C,S,?,? (especially, C,S,P,D ), C,I,?,? (especially, C,I,S,D ) have attracted more attention of researchers. It is expected that more efficient hybrids utilizing the pattern C,C,?,? will emerge in the future. For example, the component-level adaptive or self-adaptive alternation of parents, which are mentioned in Section IV-A (see Fig. 2 ), is a promising research line as it enriches the diversity of evolutionary operations to a large extent. Besides, the cooperative coevolution (CC) that is often employed for high-dimensional numerical optimization is an instance of the hybridization strategy C,C,S,D BIB002 , BIB010 . However, it has merely been successfully implemented on homogeneous optimizers, such as multiple PSOs BIB002 , BIB010 and multiple DEs BIB009 . It is possible that the CC's efficiency can be enhanced by the hybridization of diverse optimizers that have different features in explorationexploitation patterns and the Tr:Er&Ei. In addition, the hybridization of global search (especially, POs) and LS methods at component level is also a very interesting topic for future research. 2) Exploration Pattern in Memetic Algorithms Can be Enriched: Although the concept of MAs has been extended with its global search being achieved by any explorative PO BIB011 , the current focus of MAs is on the coevolution (adaptation) of different LS methods (LSMs), i.e., individual learning methods BIB004 , BIB003 , BIB006 ,  , BIB011 , BIB008 , which takes into account the diversity of LSMs and the improvement in exploitation. However, the adaptation of global search methods (GSMs) within the MA framework, i.e., the coevolution of multiple GSMs, is seldom considered, which may limit the exploration capability of MAs. A sophisticated tuning of the Tr:Er&Ei in MAs is still worth investigation. In fact, MAs usually represent a paradigm of achieving a better Tr:Er&Ei by combining GSMs and LSMs in a deterministic alternation, though some recent MA researches have provided new schemes to balance evolution (global search) and individual learning (LS) BIB011 , BIB008 . 3) Typical Hybridization Frameworks Beyond Memetic Algorithms: In contrast to MAs, parent optimizers within some different hybridization frameworks  , BIB005 ,  , regardless of their explorative or exploitative nature, will be chosen through competition or switch adaptively, running in the form of adaptive alternation. In particular, the hyperheuristics BIB001 ,  , which is also termed as superheuristics  , represent a hierarchical hybridization framework, which employs a highlevel methodology (e.g., machine-learning techniques and evolutionary algorithms) to schedule a set of low-level heuristics. Different low-level heuristics will be applied at any given time, depending on the current problem state or search stage  . An important motivation behind hyperheuristics is to devise an advanced algorithm (system), which can handle classes of problems rather than to solve only one problem, by combining the strength and compensating for the weakness of known heuristics  , BIB007 . The choice of low-level heuristics can also be encoded and evolved in a self-adaptive manner (see Fig. 2 ). Obviously, hybridization in hyperheuristics involves both assistance and collaboration. The high-level methodology serves as an assistant/scheduler of all low-level heuristics, while these heuristics collaborate with each other in search space. It is expected that a sophisticated ensemble based on the hyperheuristic framework can be built to hybridize and coordinate different explorative or/and exploitative optimizers for a wide range of problem solving.