Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> Hybrid metaheuristics have received considerable interest these recent years in the field of combinatorial optimization. A wide variety of hybrid approaches have been proposed in the literature. In this paper, a taxonomy of hybrid metaheuristics is presented in an attempt to provide a common terminology and classification mechanisms. The taxonomy, while presented in terms of metaheuristics, is also applicable to most types of heuristics and exact optimization algorithms. ::: ::: As an illustration of the usefulness of the taxonomy an annoted bibliography is given which classifies a large number of hybrid approaches according to the taxonomy. <s> BIB001 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> Preface. Acknowledgments. 1. Introduction. 2. The Gambler's Ruin and Population Sizing. 3. Master-Slave Parallel GAs. 4. Bounding Cases of GAs With Multiple Demes. 5. Markov Chain Models of Multiple Demes. 6. Migration Rates and Optimal Topologies. 7. Migration and Selection Pressure. 8. Fine-Grained and Hierarchical Parallel GAs. 9. Summary, Extensions, and Conclusions. References. Index. <s> BIB002 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> The paper presents a framework that unifies various search mechanisms for solving constrained nonlinear programming (NLP) problems. These problems are characterized by functions that are not necessarily differentiable and continuous. Our proposed framework is based on the first-order necessary and sufficient condition for constrained local minimization in discrete space that shows the equivalence between discrete-neighborhood saddle points and constrained local minima. To look for discrete-neighborhood saddle points, we formulate a discrete constrained NLP in an augmented Lagrangian function and study various mechanisms for performing ascents of the augmented function in the original-variable subspace and descents in the Lagrange-multiplier subspace. Our results show that CSAGA, a combined constrained simulated annealing (SA) and genetic algorithm (GA), performs well. Finally, we apply iterative deepening to determine the optimal number of generations in CSAGA and show that performance is robust with respect to changes in population size. <s> BIB003 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> A hybrid particle swarm with differential evolution operator, termed DEPSO, which provide the bell-shaped mutations with consensus on the population diversity along with the evolution, while keeping the self-organized particle swarm dynamics, is proposed. Then it is applied to a set of benchmark functions, and the experimental results illustrate its efficiency. <s> BIB004 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> The field of metaheuristics for the application to combinatorial optimization problems is a rapidly growing field of research. This is due to the importance of combinatorial optimization problems for the scientific as well as the industrial world. We give a survey of the nowadays most important metaheuristics from a conceptual point of view. We outline the different components and concepts that are used in the different metaheuristics in order to analyze their similarities and differences. Two very important concepts in metaheuristics are intensification and diversification. These are the two forces that largely determine the behavior of a metaheuristic. They are in some way contrary but also complementary to each other. We introduce a framework, that we call the I&D frame, in order to put different intensification and diversification components into relation with each other. Outlining the advantages and disadvantages of different metaheuristic approaches we conclude by pointing out the importance of hybridization of metaheuristics as well as the integration of metaheuristics and other methods for optimization. <s> BIB005 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> Abstract This paper presents the application of particle swarm optimization (PSO) technique and its variants to least-cost generation expansion planning (GEP) problem. The GEP problem is a highly constrained, combinatorial optimization problem that can be solved by complete enumeration. PSO is one of the swarm intelligence (SI) techniques, which use the group intelligence behavior along with individual intelligence to solve the combinatorial optimization problem. A novel ‘virtual mapping procedure’ (VMP) is introduced to enhance the effectiveness of the PSO approaches. Penalty function approach (PFA) is used to reduce the number of infeasible solutions in the subsequent iterations. In addition to simple PSO, many variants such as constriction factor approach (CFA), Lbest model, hybrid PSO (HPSO), stretched PSO (SPSO) and composite PSO (C-PSO) are also applied to test systems. The differential evolution (DE) technique is used for parameter setting of C-PSO. The PSO and its variants are applied to a synthetic test system of five types of candidate units with 6- and 14-year planning horizon. The results obtained are compared with dynamic programming (DP) in terms of speed and efficiency. <s> BIB006 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> The particle swarm optimizer (PSO) is a stochastic, population-based optimization technique that can be applied to a wide range of problems, including neural network training. This paper presents a variation on the traditional PSO algorithm, called the cooperative particle swarm optimizer, or CPSO, employing cooperative behavior to significantly improve the performance of the original algorithm. This is achieved by using multiple swarms to optimize different components of the solution vector cooperatively. Application of the new PSO algorithm on several benchmark optimization problems shows a marked improvement in performance over the traditional PSO. <s> BIB007 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> In this paper differential evolution is extended by using the notion of speciation for solving multimodal optimization problems. The proposed species-based DE (SDE) is able to locate multiple global optima simultaneously through adaptive formation of multiple species (or subpopulations) in an DE population at each iteration step. Each species functions as an DE by itself. Successive local improvements through species formation can eventually transform into global improvements in identifying multiple global optima. In this study the performance of SDE is compared with another recently proposed DE variant CrowdingDE. The computational complexity of SDE, the effect of population size and species radius on SDE are investigated. SDE is found to be more computationally efficient than CrowdingDE over a number of benchmark multimodal test functions. <s> BIB008 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> Adaptation of parameters and operators represents one of the recent most important and promising areas of research in evolutionary computations; it is a form of designing self-configuring algorithms that acclimatize to suit the problem in hand. Here, our interests are on a recent breed of hybrid evolutionary algorithms typically known as adaptive memetic algorithms (MAs). One unique feature of adaptive MAs is the choice of local search methods or memes and recent studies have shown that this choice significantly affects the performances of problem searches. In this paper, we present a classification of memes adaptation in adaptive MAs on the basis of the mechanism used and the level of historical knowledge on the memes employed. Then the asymptotic convergence properties of the adaptive MAs considered are analyzed according to the classification. Subsequently, empirical studies on representatives of adaptive MAs for different type-level meme adaptations using continuous benchmark problems indicate that global-level adaptive MAs exhibit better search performances. Finally we conclude with some promising research directions in the area. <s> BIB009 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> Penalty function based PSO converts constrained optimization problems into non-constrained optimization problems, but slow convergence and premature convergence easily happen because of inappropriate penalty coefficients. Modified PSO by tracking best feasible particle can not facilitate particles exploring unknown feasible region from known infeasible region, so the global exploration ability is greatly limited. Therefore, finding better unknown feasible solution by flying through infeasible region is critical to the performance of PSO. This paper proposes Master-Slave Particle Swarm Optimization (MSPSO), a novel approach for solving constrained optimization problems, in which particles in master swarm fly toward better feasible particles, particles in slave swarm fly toward better infeasible particles, and particles in two swarms help each other flying by sharing information of better feasible and infeasible particles. The proposed algorithm was tested on 11 benchmark constrained optimization problems. The test results show that MSPSO can significantly improve the globe exploration ability and effectively avoid being trapped into local optimum. By comparison with other evolutionary algorithms, MSPSO performs better for constrained optimization problems. <s> BIB010 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> In this correspondence, an approach based on coevolutionary particle swarm optimization to solve constrained optimization problems formulated as min-max problems is presented. In standard or canonical particle swarm optimization (PSO), a uniform probability distribution is used to generate random numbers for the accelerating coefficients of the local and global s. We propose a Gaussian probability distribution to generate the accelerating coefficients of PSO. Two populations of PSO using Gaussian distribution are used on the optimization algorithm that is tested on a suite of well-known benchmark constrained optimization problems. Results have been compared with the canonical PSO (constriction factor) and with a coevolutionary genetic algorithm. Simulation results show the suitability of the proposed algorithm in terms of effectiveness and robustness <s> BIB011 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> Many scientific and engineering applications require optimization methods to find more than one solution to multimodal optimization problems. This paper presents a new particle swarm optimization (PSO) technique to locate and refine multiple solutions to such problems. The technique, NichePSO, extends the inherent unimodal nature of the standard PSO approach by growing multiple swarms from an initial particle population. Each subswarm represents a different solution or niche; optimized individually. The outcome of the NichePSO algorithm is a set of particle swarms, each representing a unique solution. Experimental results are provided to show that NichePSO can successfully locate all optima on a small set of test functions. These results are compared with another PSO niching algorithm, lbest PSO, and two genetic algorithm niching approaches. The influence of control parameters is investigated, including the relationship between the swarm size <s> BIB012 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> A new, almost parameter-free optimization algorithm is developed in this paper as a hybrid of the barebones particle swarm optimizer (PSO) and differential evolution (DE). The DE is used to mutate, for each particle, the attractor associated with that particle, defined as a weighted average of its personal and neighborhood best positions. Results of this algorithm are compared to that of the barebones PSO, Von Neumann PSO, a DE PSO, and DE/rand/1/bin. These results show that the new algorithm provides excellent results with the added advantage that no parameter tuning is needed <s> BIB013 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> This paper proposes the super-fit memetic differential evolution (SFMDE). This algorithm employs a differential evolution (DE) framework hybridized with three meta-heuristics, each having different roles and features. Particle Swarm Optimization assists the DE in the beginning of the optimization process by helping to generate a super-fit individual. The two other meta-heuristics are local searchers adaptively coordinated by means of an index measuring quality of the super-fit individual with respect to the rest of the population. The choice of the local searcher and its application is then executed by means of a probabilistic scheme which makes use of the generalized beta distribution. These two local searchers are the Nelder mead algorithm and the Rosenbrock Algorithm. The SFMDE has been tested on two engineering problems; the first application is the optimal control drive design for a direct current (DC) motor, the second is the design of a digital filter for image processing purposes. Numerical results show that the SFMDE is a flexible and promising approach which has a high performance standard in terms of both final solutions detected and convergence speed. <s> BIB014 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> A chaotic co-evolutionary algorithm based on differential evolution and particle swarm optimization (CCDEPSO) is proposed. CCDEPSO is a two-population co-evolutionary algorithm, in which the individuals of one population are evolved according to differential evolution algorithm (DE) and the other individuals are evolved according to particle swarm optimization algorithm (PSO). In order to realize co-evolving of the two sub-populations, an information sharing scheme by sharing the fitness value and the corresponding position value of the two sub-populations is introduced. Furthermore, in order to improve the searching speed and avoid the results trapping in local optima prematurely, chaotic initialization and chaotic perturbation based on Tent map are introduced in CCDEPSO. The comparative testing experiments are performed by means of CCDEPSO, DE, CPSO and PSO algorithms on six benchmark functions. Experimental results demonstrate that the global optimization ability of CCDEPSO is better than other algorithms above mentioned. <s> BIB015 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> Several techniques have been proposed to extend the particle swarm optimization (PSO) paradigm so that multiple optima can be located and maintained within a convoluted search space. A significant number of these implementations are subswarm-based, that is, portions of the swarm are optimized separately. Niches are formed to contain these subswarms, a process that often requires user-specified parameters. The proposed technique, known as the vector-based PSO, uses a novel approach to locate and maintain niches by using additional vector operations to determine niche boundaries. As the standard PSO uses weighted vector combinations to update particle positions and velocities, the niching technique builds upon existing knowledge of the particle swarm. Once niche boundaries have been calculated, the swarm can be organized into subswarms without prior knowledge of the number of niches and their corresponding niche radii. This paper presents the vector-based PSO with emphasis on its underlying principles. Results for a number of functions with different characteristics are reported and discussed. The performance of the vector-based PSO is also compared to two other niching techniques for particle swarm optimization. <s> BIB016 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> This paper introduces a new vector evaluated multi-objective optimization algorithm. The vector evaluated differential evolution particle swarm optimization (VEDEPSO) algorithm is a hybridization of the classical vector evaluated particle swarm optimization (VEPSO) and vector evaluated differential evolution (VEDE) algorithms of Parsopoulos et. al. [9], [10]. Comparisons of VEDEPSO with respect to VEPSO and VEDE on a well known multi-objective benchmark problem set indicated that significant performance improvements can be attributed to the VEDEPSO algorithm. <s> BIB017 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> in the present study we propose a new hybrid version of Differential Evolution (DE) and Particle Swarm Optimization (PSO) algorithms. In the proposed algorithm named as Hybrid Differential Evolution (HDE) a ‘switchover constant’ called α is defined. HDE starts as the basic DE algorithm which switches over to PSO when α is activated. The constant α on the other hand is activated at a point where the DE procedure usually slows down which is usually in the proximity of global optimal i.e. to say when the search domain is contracted around the global optimal. Experimenting with a test bed of 12 benchmark functions, we show the promising nature of the proposed HDE algorithm. <s> BIB018 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> This paper presents extensive experiments on a hybrid optimization algorithm (DEPSO) we recently developed by combining the advantages of two powerful population-based metaheuristics—differential evolution (DE) and particle swarm optimization (PSO). The hybrid optimizer achieves on-the-fly adaptation of evolution methods for individuals in a statistical learning way. Two primary parameters for the novel algorithm including its learning period and population size are empirically analyzed. The dynamics of the hybrid optimizer is revealed by tracking and analyzing the relative success ratio of PSO versus DE in the optimization of several typical problems. The comparison between the proposed DEPSO and its competitors involved in our previous research is enriched by using multiple rotated functions. Benchmark tests involving scalability test validate that the DEPSO is competent for the global optimization of numerical functions due to its high optimization quality and wide applicability. <s> BIB019 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> This article proposes a distributed differential evolution which employs a novel self-adaptive scheme, namely scale factor inheritance. In the proposed algorithm, the population is distributed over several sub-populations allocated according to a ring topology. Each sub-population is characterized by its own scale factor value. With a probabilistic criterion, that individual displaying the best performance is migrated to the neighbor population and replaces a pseudo-randomly selected individual of the target sub-population. The target sub-population inherits not only this individual but also the scale factor if it seems promising at the current stage of evolution. In addition, a perturbation mechanism enhances the exploration feature of the algorithm. The proposed algorithm has been run on a set of various test problems and then compared to two sequential differential evolution algorithms and three distributed differential evolution algorithms recently proposed in literature and representing state-of-the-art in the field. Numerical results show that the proposed approach seems very efficient for most of the analyzed problems, and outperforms all other algorithms considered in this study. <s> BIB020 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> 1) Hybridization Level: <s> Abstract To solve the highly constrained environmental/economic dispatch problem involving conflicting objectives, this paper presents a hybrid multi-objective optimization algorithm based on particle swarm optimization (PSO) and differential evolution (DE). In this algorithm, a PSO with time variant acceleration coefficients is designed to explore the entire search space, while a local version of DE is proposed to exploit the sub-space with sparse solutions. A crowing distance-based approach is introduced to assign the particles’ leaders and to update the external archive. Moreover, a new technique for equality constraints is proposed to hurdle the unfeasible solutions directly. Finally, several optimization trials of the proposed algorithm are carried out on the IEEE 30-bus test system. Results demonstrate superiority of the proposed approach and confirm its potential to solve the multi-objective EED problem. <s> BIB021
The HL of parent optimizers, which are highly dependent on their respective operating levels (OLs), refers to the OL at which they are hybridized, indicating the spatial assignment of parent optimizers. As illustrated in Fig. 1 , there are four kinds of HLs-component level, individual level, subpopulation level, and population level (in order from the lowest to the highest level). This classification is also applicable to the OL. The OL of a parent optimizer is determined by the object, which can, according to hybridization design, apply the optimizer without being required to use the same optimizer with any other objects. If the object is a whole population, we say the parent operates at population level. Likewise, a parent operates at subpopulation level, individual level, or component level, if the corresponding object is a subpopulation, an individual, or a solution component (or multiple components), respectively. Operating at component level, a parent optimizer will be used to regulate one or more components of candidate solutions rather than an entire solution (individual). Besides, different components of an individual can be evolved by different methods, that is to say, the individual will be evolved by multiple methods simultaneously. In this case, designers may specify some evolution methods for different components or grant them the freedom to opt for their evolution methods according to some indices (e.g., some predefined selection probability). DEPSO-HGH provides an example of the hybrid whose parents operate at component level [see (11)- (13)]. In contrast, each time a parent optimizer works, operating at individual level, it will evolve an entire solution (individual), but usually not all solutions in the population. In this case, the individual can independently apply the parent optimizer without being required to adopt the same evolution method with any other individuals. The individual may have the freedom to choose its evolution methods according to some selection rules. For example, the individuals in DEPSO-XCPP independently choose their own evolution methods (PSO or DE) according to the success ratio of candidate methods BIB019 . If a parent optimizer operates at population level, the evolution method once applied will simultaneously act on all individuals in the population; howbeit, different methods may work on the whole population alternately at different generations. For example, the parents of DEPSO-ZX operate at population level since its PSO parent evolves the entire population and DE follows to further improve the personal best positions of all particles BIB004 . As an intermediate case between individual and population levels, subpopulation level indicates a division of the whole population, which is often termed as multispecies strategy or multiswarm strategy (aka, the niching technique in GAs research) BIB002 . Parents that operate at subpopulation level will be applied to different subpopulations. DEPSO-ZZS provides an example of the hybridization at subpopulation level BIB015 . It is worthwhile to mention that the niching technique is usually applied to homogeneous optimizers, such as GAs, giving birth to many niching-based EAs (e.g., the niching GA BIB002 , niching PSO BIB016 , BIB012 , and niching DE BIB020 , BIB008 ). From a broader perspective, hybridization with parents that operate at subpopulation level can be deemed as an extension of the niching technique (see also the definition of homogeneity that is given by Talbi BIB001 ). Particularly, as for multiobjective optimization, the parent optimizers can be assigned to different objectives with each parent manipulating its own subpopulation (e.g., DEPSO-GE BIB017 ). The subpopulation level with appropriate division of labor based on specialization of parents is also a nice choice to design a hybrid to solve constrained optimization problems or track multiple optima in multimodal landscapes. If parents work at the same OL, we term the resultant hybridization as homogeneous-level hybridization (HOLH), and the OL of parents is regarded as their HL. In contrast, heterogeneous-level hybridization (HELH) means that the OLs of parents are different. For example, regarding the hybridization between a PO and a trajectory search method (TSM), i.e., singlepoint-based optimizer BIB005 , in which the PO usually operates at population level, the TSM will be regarded to be hybridized with the PO at individual level if a TSM manipulates only one solution at each generation. As a rule, the HL in HELH between two optimizers is determined by the lower OL of parents. Furthermore, some special HOLH can be termed as homogeneousobject hybridization (HOOH), if parents share the same object to determine their OLs, which means that they are employed to evolve the same object at their HL. In contrast, heterogeneousobject hybridization (HEOH) implies that the objects, which determine the OLs of parents are different even though the objects may be of the same type (e.g., subpopulation type). For example, parents that operate on different subpopulations give birth to HEOH (e.g., DEPSO-ZZS BIB015 ). Besides, undoubtedly we can categorize any HELH as HEOH. 2) Operating Order: The OO of parent optimizers refers to the temporal assignment of parent optimizers. Parents can work in a sequential (alternate) or parallel order. Talbi terms the sequential OO type as relay, meaning that a set of metaheuristics is applied one after another, each using the output of the previous as its input, acting in a pipeline fashion BIB001 . Das categorizes hybrids based on the two OO types as serial hybridization and parallel hybridization, respectively  . Particularly, parents of embedding-based hybrids work together as an ensemble, regardless of the OO. Das terms embedding-based hybrids as implicit hybridization  . For clarity of classification, we term the case of being unable to distinguish the order of parents as "no order". The OO regarding a hybrid should be canonically analyzed at its HL, which is a crucial rule to identify the OO of parents since different angles of view may lead to different conclusions with respect to OO. For example, the parents in DEPSO-OES1 BIB013 , from a population perspective, hold a parallel OO as they can simultaneously operate on different individuals at each generation; howbeit, they act alternately on any single individual from the perspective of their HL (i.e., individual level). To avoid confusion, we will treat the analysis of OO regarding HOOH and HEOH in different ways. For HOOH, parents hold a parallel OO if and only if they can simultaneously operate on the same object that determines their OLs. If the parents evolve the object in a "relay" manner as Talbi claimed BIB001 , we say they hold a sequential OO. For HEOH, parents usually hold a parallel OO as they operate on different objects, facilitating the implementation of parallel OO. However, some special interdependence between parents may force them to carry out a sequential OO. For example, in assistance-based hybrids, the master parent and its assistant usually operate on different objects (e.g., in DEPSO-KSSP BIB006 , a population of parameter vectors for DE and a solution population for PSO), and the parents definitely hold a sequential OO since the master relies on the information that is supplied by its assistant to implement its variables p and p i , i ∈ {1, 2, . . . , D}, represent the choice of parent optimizers for the evolution of an individual and a component, respectively. Taking DEPSO for an example, the variables can be set to PSO (0) or DE (1). The extended individual X will undergo the evolutionary operations. As opposed to the individual-level self-adaptation, the component-level self-adaptation brings more diversity of evolutionary operations. The quality of the extended individuals is evaluated by the fitness of corresponding real solutions. It is implicitly acknowledged that high-quality solutions come along with appropriate evolution methods. operations. Another example is the CPSO-S K that is proposed by Bergh and Engelbrecht BIB007 in which multiple homogeneous PSOs (parents) operate on different solution components and share them through a context vector for fitness evaluation. The PSO parents will one by one, rather than simultaneously, evolve their respective swarm and update the corresponding solution components of the context vector. The sequential OO, here, is advantageous to restrain different PSOs from the utilization of the same context vector for fitness evaluation and yield a finegrained search BIB007 . As for the sequential OO, the alternation of parents, viz., choosing parents to run at different intervals, can be achieved in different ways. Referring to the assortment of alternation, it is worthwhile to mention the classification of adaptive MAs that are proposed by Ong et al. BIB009 . The classification takes into account two criterions to sort different adaptive MAs: adaptation type (static/adaptive/self-adaptive) and adaptation level (external/local/global). The adaptation type is identified to be static if the feedback from the search process regarding the online performance of different memes (i.e., local IMs) is not utilized for the choice of memes. In contrast, both adaptive and self-adaptive types utilize the feedback to guide the choice of memes. However, the self-adaptive type integrates (encodes) the choice of memes into the representation of individual solutions. The conceptually extended "individual" involves not only a real solution but also its local IM (meme). Fig. 2 is an illustration of the individual-level self-adaptation and the component-level self-adaptation. The adaptation level refers to the extent of prior knowledge about memes. External-level adaptation usually depends on the experiences of human experts, instead of online knowledge, to choose the memes. Local-level and global-level adaptations, which are differentiated by the range of utilized historical knowledge, choose the memes according to their online performance. The aforementioned classification is also applicable to different types of alternation since the alternation of parent optimizers is de facto an adaptation behavior. In the following, we provide a subdivision of static alternation and adaptive alternation to reveal more design issues regarding hybrids based on the sequential OO. Static alternation can take different forms, such as deterministic alternation and stochastic alternation. The deterministic alternation means that parents will alternate according to a predefined deterministic order. Therefore, the timetable by which each parent runs and terminates is prepared in advance. For example, in the two-phase DEPSO-CNT, PSO runs first, providing DE with the so-called super-fit individual BIB014 . In DEPSO-ZX, PSO, and DE also alternate regularly in a deterministic order BIB004 . Usually, the deterministic execution order of parents stems from the insight and experience of designers. For example, the idea behind DEPSO-APA is that DE/rand/1/bin is more suitable for early-stage exploration than the basic PSO (Gbest topology), while the latter is more competent for exploitation BIB018 . Stochastic alternation means that the execution of parents is a random event that is dependent on the selection probability of different parents, which may be constant or change with evolution stages. The setting of the selection probability that pertains to static alternation is also a pure human factor, which is unrelated to the online performance of parents. Regarding adaptive alternation, we can adopt various schemes of utilization of feedback information to conduct the selection of parents in the alternation process. The evolution process can switch from one parent to another when it is identified that the in-service parent cannot bring fitness improvement. This scheme seems conservative as it does not care about the performance difference of parents. In contrast, a greedy scheme only chooses the parent, which performs the best according to search history, resulting in a strong competition among parents. A moderate alternative is to adjust the selection probability of parents in light of their relative performance. For example, the PSO and DE in DEPSO-XCPP are adaptively chosen according to their relative success ratio of improvement of the fitness of individuals BIB019 . To date, how to utilize the feedback from search process to choose competent parents is still an open issue. For the parallel OO, attention should be paid to the interaction of parents, including the information shared between parents and the occasions on which the information is shared. The parallel order can be implemented with the support of hardware architecture being capable of achieving parallel computing (e.g., single-instruction stream with multiple-data stream and multiple-instruction stream with multiple-data stream). A further discussion on the parallel architecture for the implementation of hybrid metaheuristics can be found in BIB001 . 3) Type of Information Transfer: The TIT refers to the interconnection structure of parent optimizers, concerning the information flow between them. By regarding each parent as an "individual (particle)," we can equate the TIT with the population topology of parent optimizers. Fig. 3 provides four typical interconnection structures. Pertaining to the hybridization between two parent optimizers, what concerns the classification of the TIT is the direction of information flow, either unidirectional (simplex TIT) or bidirectional (duplex TIT). Generally speaking, the duplex TIT leads to a stronger coupling between parents than the simplex TIT. Most existing DEPSOs employ the duplex TIT, manifesting that the interactive influence between DE and Fig. 3 . Typical interconnection structures: Arrows represent the direction of the information flows between two parents. A bidirectional arrow indicates that the related parents transfer information in a duplex manner. In the structure that is shown on the far left, Parent P1 works independently since it does not receive any information from the other two parents. The behavior of P2 is affected by P1 as P2 utilizes the information from P1 to guide its search. In the second structure, P1 and P2 interact and jointly guide the search of P3. The third structure looks like the well-known Ring topology in PSO research. Moreover, it defines the information flow between parents, giving birth to a unidirectional flow of the mutual influence of parents. The final cube structure is termed as the island model in niching GA research BIB001 , BIB002 . PSO has drawn more attention from DEPSO designers. DEPSO-CNT BIB014 , DEPSO-APA BIB018 , and DEPSO-KSSP BIB006 are the three examples of the simplex TIT. In addition, if the parents in a duplex TIT transfer information of the same type, the TIT will be termed as the homogeneous TIT, and heterogeneous TIT otherwise. It should be stressed that the goal of information transfer in hybridization is the information utilization that is linked to interactions between parents. If the transferred information is merely shared but not utilized to affect the search behavior of other parents, the parents will work independently. In this case, the hybridization without interactions cannot bring the expected performance emergence since it can be easily derived that where AB is the hybrid of the parents A and B, Perf AB (m + n) is the performance index of AB with m + n function evaluations (FEs), andPerf A (m) and Perf B (n) are the performance indices of A with m FEs and B with n FEs, respectively. The aforementioned inequality holds for the optimization of any problem, indicating that the hybrid AB cannot outperform both parents at the same time. Therefore, the hybridization without interactions is meaningless and the utilization of transferred information is essential. To conclude, no interaction, no hybridization! 4) Type of Transferred Information: As a crucial design element of hybrid optimizers, the TTI is the most complicated factor among the aforementioned hybridization factors. It is not only related to specific optimizers, but also to the type of the problems to be solved. Common TTIs include current solutions (aka, target vectors for DE and current particle positions for PSO), group of elite solutions, so-far-best solution (gbest), solution components, and control parameters. Regarding PSO, the alternative for the TTI can also be pbest, nbest, or particle velocities. For DE, its base vectors, difference vectors, mutant vectors, and trial vectors can be the candidate TTIs as well. Besides, the transferred information may be a combination of the TTIs mentioned earlier. As for multiobjective optimization, the external archive that is used to store high-quality (Pareto optimal) nondominated solutions is also a common TTI (e.g., DEPSO-GZQ BIB021 and DEPSO-GE BIB017 ). For constrained optimization, high-quality feasible and infeasible solutions can be treated as the candidate TTIs. For example, Yang et al. proposed a master-slave PSO algorithm to solve constrained optimization problems with two PSOs, a master and a slave, manipulating their own subpopulations and cooperatively updating a feasible leader set (FLS) and an infeasible leader set (ILS) BIB010 . The FLS and ILS are expected to record elite feasible solutions and high-quality infeasible solutions, respectively. The master PSO and the slave choose nbest from the FLS and ILS, respectively. Resembling the master-slave PSO, a new DEPSO for constrained optimization can be constructed by enabling DE and PSO to be guided by FLS and ILS, respectively (vice versa). Some special TTIs can be introduced by specific approaches. For example, in the coevolutionary PSO algorithm that is proposed by Krohling and Coelho to solve constraint optimization problems BIB011 , two PSOs operate on real solutions and Lagrange multipliers, respectively. The Lagrange multipliers are introduced by a Lagrange approach, which formulates constrained optimization problems into min-max problems. The two PSOs, which are hybridized in a sequential OO, share and utilize the real solutions and the Lagrange multipliers during fitness evaluations. A similar instance is the hybrid of SA and GA for nonlinear constrained optimization BIB003 . From the earlier discussion, it can be seen that the TTI is a complicated hybridization factor with plentiful options, making it very hard to enumerate all TTIs. Instead, a coarse-grained classification of the TTI is provided, covering the following TTI types: 1) solutions (e.g., gbest, pbest, nbest, DE's target vectors and trial vectors, the archive of nondominated solutions, and the FLS and ILS mentioned earlier); 2) fitness information; 3) solution components; 4) algorithm-induced in-betweens (e.g., DE's base vectors, mutant vectors, difference vectors, PSO's particle velocities); 5) auxiliaries (e.g., Lagrange multipliers); and 6) control parameters. The difference between solutions and algorithm-induced inbetweens lies in that solutions are expected to have gone through fitness evaluation but algorithm-induced in-betweens are merely intermediate variables that are introduced by specific optimizers and usually taken as ingredients for generation of new solutions.