Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Assistance-Based DEPSOs <s> The task of optimizing a complex system presents at least two levels of problems for the system designer. First, a class of optimization algorithms must be chosen that is suitable for application to the system. Second, various parameters of the optimization algorithm need to be tuned for efficiency. A class of adaptive search procedures called genetic algorithms (GA) has been used to optimize a wide variety of complex systems. GA's are applied to the second level task of identifying efficient GA's for a set of numerical optimization problems. The results are validated on an image registration problem. GA's are shown to be effective for both levels of the systems optimization problem. <s> BIB001 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Assistance-Based DEPSOs <s> The particle swarm algorithm has just enough moving parts to make it hard to understand. The formula is very simple, it is even easy to describe the working of the algorithm verbally, yet it is very difficult to grasp in one's mind how the particles oscillate around centers that are constantly changing; how they influence one another; how the various parameters affect the trajectory of the particle; how the topology of the swarm affects its performance; and so on. This paper strips away some traditional features of the particle swarm in the search for the properties that make it work. The particle swarm algorithm is modified by eliminating the velocity formula. Variations are compared. In the process some of the mysteries of the algorithm are revealed, we discover its similarity to other stochastic population-based problem solving methods, and new avenues of investigation are suggested or implied. <s> BIB002 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Assistance-Based DEPSOs <s> Abstract This paper presents the application of particle swarm optimization (PSO) technique and its variants to least-cost generation expansion planning (GEP) problem. The GEP problem is a highly constrained, combinatorial optimization problem that can be solved by complete enumeration. PSO is one of the swarm intelligence (SI) techniques, which use the group intelligence behavior along with individual intelligence to solve the combinatorial optimization problem. A novel ‘virtual mapping procedure’ (VMP) is introduced to enhance the effectiveness of the PSO approaches. Penalty function approach (PFA) is used to reduce the number of infeasible solutions in the subsequent iterations. In addition to simple PSO, many variants such as constriction factor approach (CFA), Lbest model, hybrid PSO (HPSO), stretched PSO (SPSO) and composite PSO (C-PSO) are also applied to test systems. The differential evolution (DE) technique is used for parameter setting of C-PSO. The PSO and its variants are applied to a synthetic test system of five types of candidate units with 6- and 14-year planning horizon. The results obtained are compared with dynamic programming (DP) in terms of speed and efficiency. <s> BIB003 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Assistance-Based DEPSOs <s> A new, almost parameter-free optimization algorithm is developed in this paper as a hybrid of the barebones particle swarm optimizer (PSO) and differential evolution (DE). The DE is used to mutate, for each particle, the attractor associated with that particle, defined as a weighted average of its personal and neighborhood best positions. Results of this algorithm are compared to that of the barebones PSO, Von Neumann PSO, a DE PSO, and DE/rand/1/bin. These results show that the new algorithm provides excellent results with the added advantage that no parameter tuning is needed <s> BIB004 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Assistance-Based DEPSOs <s> A cultural algorithm based on differential particle swarm optimization (DPSO) is proposed in this paper. The main novel feature of this approach is the use of DPSO as a population space. DPSO is a hybrid-optimized algorithm based principally on the introduction of the differential variation mechanism into the PSO algorithm, which will maintain swarm diversity. The knowledge sources contained in the belief space of the cultural algorithm are specifically designed according to the DPSO evolution population. Different knowledge sources are used to influence the variation operator of DPSO, in order to enhance search ability and reduce the computational cost. The simulation results of typical function optimization problems show that the proposed algorithm has good optimization quality. <s> BIB005 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Assistance-Based DEPSOs <s> The fast convergence of particle swarm algorithms can become a downside in multi-objective optimization problems when there are many local optimal fronts. In such a situation a multi-objective particle swarm algorithm may get stuck to a local Pareto optimal front. In this paper we propose a new approach in selecting leaders for the particles to follow, which in-turn will guide the algorithm towards the Pareto optimal front. The proposed algorithm uses a Differential Evolution operator to create the leaders. These leaders can successfully guide the other particles towards the Pareto optimal front for various types of test problems. This simple yet robust algorithm is effective compared with existing multi-objective particle swarm algorithms. <s> BIB006 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Assistance-Based DEPSOs <s> The barebones differential evolution (BBDE) is a new, almost parameter-free optimization algorithm that is a hybrid of the barebones particle swarm optimizer and differential evolution. Differential evolution is used to mutate, for each particle, the attractor associated with that particle, defined as a weighted average of its personal and neighborhood best positions. The performance of the proposed approach is investigated and compared with differential evolution, a Von Neumann particle swarm optimizer and a barebones particle swarm optimizer. The experiments conducted show that the BBDE provides excellent results with the added advantage of little, almost no parameter tuning. Moreover, the performance of the barebones differential evolution using the ring and Von Neumann neighborhood topologies is investigated. Finally, the application of the BBDE to the real-world problem of unsupervised image classification is investigated. Experimental results show that the proposed approach performs very well compared to other state-of-the-art clustering algorithms in all measured criteria. <s> BIB007 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Assistance-Based DEPSOs <s> An algorithm called DE-PSO is proposed which incorporates concepts from DE and PSO, updating particles not only by DE operators but also by mechanisms of PSO. The proposed algorithm is tested on several benchmark functions. Numerical comparisons with different hybrid meta-heuristics demonstrate its effectiveness and efficiency. <s> BIB008 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Assistance-Based DEPSOs <s> Differential evolution (DE) is well known as a simple and efficient scheme for global optimization over continuous spaces. It has reportedly outperformed a few evolutionary algorithms (EAs) and other search heuristics like the particle swarm optimization (PSO) when tested over both benchmark and real-world problems. DE, however, is not completely free from the problems of slow and/or premature convergence. This paper describes a family of improved variants of the DE/target-to-best/1/bin scheme, which utilizes the concept of the neighborhood of each population member. The idea of small neighborhoods, defined over the index-graph of parameter vectors, draws inspiration from the community of the PSO algorithms. The proposed schemes balance the exploration and exploitation abilities of DE without imposing serious additional burdens in terms of function evaluations. They are shown to be statistically significantly better than or at least comparable to several existing DE variants as well as a few other significant evolutionary computing techniques over a test suite of 24 benchmark functions. The paper also investigates the applications of the new DE variants to two real-life problems concerning parameter estimation for frequency modulated sound waves and spread spectrum radar poly-phase code design. <s> BIB009
Kannan et al. BIB003 proposed a distinctive DEPSO (namely DEPSO-KSSP; C-PSO in BIB003 ). Its PSO parent takes the form of classical PSO [cf., (6) and (7)]. The DE algorithm in it is employed to select three control parameters (i.e., w, c1, and c2) online for PSO. In other words, DE serves as a metaoptimizer for the optimization of PSO's search behavior. The DE population is randomly initialized and, then, evolves along with particle swarm. The evaluation of a DE individual (i.e., a PSO parameter setting) is attached to the fitness evaluation of the corresponding particle. Denote by para i = [w i , c1 i , c2 i ] T the ith DE individual. Then, the primary operations of DE in DEPSO-KSSP can be described as follows. Differential mutation: Crossover: The idea to employ a metaoptimizer to optimize (regulate) the search behavior of another optimizer used to be adopted by Grefenstette for the optimization of GA's control parameters BIB001 . In Grefenstette's research, a meta-GA is employed to tune the parameters of another GA. Jiang et al. put forward a novel hybridization strategy, which uses DE to operate on the velocity of particles  . As Jiang et al. pointed out, the population diversity of PSO is highly dependent on the velocity of particles, indicating that an appropriate control scheme of the particle velocity can benefit the global convergence of PSO. The primary operations of their DEPSO (namely DEPSO-JWJ) include basic PSO operations [cf., (6) and (7)] and the following: (27) , we can term the DE parent of DEPSO-JWJ as DE/current/2/bin by following the nomenclature that is mentioned in Section II-A. The operations that are shown in (27) and (28) will be implemented only if PSO is trapped into stagnation. The operation shown in (28) is not a part of DE but a BBPSO perturbed by the velocity that is generated by DE. To sum up, the DE parent in this DEPSO serves as an assistant of PSO to regulate the velocity of particles. The second DEPSO that is proposed in BIB004 by Omran et al. (namely DEPSO-OES2) combined the BBPSO that is proposed by Kennedy BIB002 and DE as follows: where p r is the predefined recombination probability, and the other denotations are the same as those in (1)- (7). If the neighborhood topology is based on the Gbest model, the term nbest in (30) will be replaced by gbest and nbest i on each dimension. Obviously, the point that is generated by BBPSO, i.e., p T , is taken as the base vector for the DE operation that is reflected by (29) . Therefore, the role of BBPSO in DEPSO-OES2 is an assistant, providing base vectors for DE. DEPSO-OES2 (named barebones DE by its inventors) was applied to unsupervised image classification BIB007 . Wickramasinghe and Li proposed an assistance-based DEPSO (namely DEPSO-WL) for multiobjective optimization by using DE/rand/1/bin to choose the leaders (i.e., nbest i ) for PSO-cf at each generation BIB006 . The trial vector that is generated by DE is taken as the leader in PSO. It should be noted that the trial vector will not undergo fitness evaluation, indicating that the selection operator of DE is removed and the trial vector is not a real sampling point. In this sense, DE is an assistant of PSO to provide the ingredient (i.e., nbest i ) for its operation. However, once the trial vector is evaluated, the DE parent will contribute to fitness improvement as a major participant in the search of high-quality solutions rather than merely an assistant of PSO, and in this case, the resultant DEPSO will be based on collaboration. Some intricate DEPSOs may involve more than one kind of parent relationship. For example, in the DEPSO that is proposed by Zhang et al. BIB008 (namely DEPSO-ZNLOD), DE and PSO on one hand are hybridized in an embedding way and, on the other hand, work together under a collaboration relationship. The DE operations in DEPSO-ZNLOD are shown as follows: It is obvious that PSO's population topology is embedded into DE by taking nbest as the base vector for differential mutation [cf., (31)]. Therefore, the aforementioned DE is actually an embedding-based DEPSO (cf., DEPSO-DACK BIB009 ). Moreover, DEPSO-ZNLOD employs the classical PSO as a method to evolve the whole population. In fact, the DE works only at specified intervals to fine tune the personal best particle positions BIB008 , and most of the time, the PSO dominates the evolutionary process. Accordingly, the DE and PSO also hold a collaboration relationship. Liu et al. incorporated DE/rand/1/bin and PSO into the framework of cultural algorithm BIB005 , and the resultant hybrid is named DEPSO-LWY here. A primary operation is shown in (36) , which reflects the hybridization of DE and PSO based on both embedding (DE's mutation is perturbed by PSO's velocity) and collaboration (PSO and perturbed DE are further hybridized on component level). Besides, unlike most PSO variants, the par- 1, 2, . . . , D) are the components of X. ticle positions and velocities are modulated according to four kinds of knowledge sources that are contained in the belief space of the cultural algorithm. Particle positions will be drawn toward the global best position (i.e., gbest), and particle velocities will be confined by lower and upper bounds that are dependent on personal best particle positions. In this sense, the PSO is further hybridized with the DEPSO that is shown in (36) under a collaboration relationship: Although the aforementioned multifarious DEPSOs were constructed with the same purpose to combine two or more excellent optimizers to create a more advanced one, they have more or less differences in their hybridization strategies and parent optimizers. A systematic taxonomy, as shown in the next section, will be beneficial to analyze different kinds of DEPSOs in a comprehensive way.