Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> This paper shows how evolutionary algorithms can be described in a concise, yet comprehensive and accurate way. A classification scheme is introduced and presented in a tabular form called TEA (Table of Evolutionary Algorithms). It distinguishes between different classes of evolutionary algorithms (e.g., genetic algorithms, ant systems) by enumerating the fundamental ingredients of each of these algorithms. At the end, possible uses of the TEA are illustrated on classical evolutionary algorithms. <s> BIB001 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> The paper analyses recent developments of a number of memory-based metaheuristics such as taboo search, scatter search, genetic algorithms and ant colonies. Its shows that the implementations of these general solving methods are more and more similar. So, a unified presentation is proposed under the name of Adaptive Memory Programming (AMP). A number of methods recently developed for the quadratic assignment, vehicle routing and graph colouring problems are reviewed and presented under the adaptive memory programming point of view. AMP presents a number of interesting aspects such as a high parallelization potential and theability of dealing with real and dynamic applications. <s> BIB002 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> The particle swarm algorithm has just enough moving parts to make it hard to understand. The formula is very simple, it is even easy to describe the working of the algorithm verbally, yet it is very difficult to grasp in one's mind how the particles oscillate around centers that are constantly changing; how they influence one another; how the various parameters affect the trajectory of the particle; how the topology of the swarm affects its performance; and so on. This paper strips away some traditional features of the particle swarm in the search for the properties that make it work. The particle swarm algorithm is modified by eliminating the velocity formula. Variations are compared. In the process some of the mysteries of the algorithm are revealed, we discover its similarity to other stochastic population-based problem solving methods, and new avenues of investigation are suggested or implied. <s> BIB003 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> Manifold possibilities of hybridizing individual metaheuristics with each other and/or with algorithms from other fields exist. A large number of publications documents the benefits and great success of such hybrids. This article overviews several popular hybridization approaches and classifies them based on various characteristics. In particular with respect to low-level hybrids of different metaheuristics, a unified view based on a common pool template is described. It helps in making similarities and different key components of existing metaheuristics explicit. We then consider these key components as a toolbox for building new, effective hybrid metaheuristics. This approach of thinking seems to be superior to sticking too strongly to the philosophies and historical backgrounds behind the different metaheuristic paradigms. Finally, particularly promising possibilities of combining metaheuristics with constraint programming and integer programming techniques are highlighted. <s> BIB004 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> A swarm algorithm framework (SWAF), realized by agent-based modeling, is presented to solve numerical optimization problems. Each agent is a bare bones cognitive architecture, which learns knowledge by appropriately deploying a set of simple rules in fast and frugal heuristics. Two essential categories of rules, the generate-and-test and the problem-formulation rules, are implemented, and both of the macro rules by simple combination and subsymbolic deploying of multiple rules among them are also studied. Experimental results on benchmark problems are presented, and performance comparison between SWAF and other existing algorithms indicates that it is efficiently. <s> BIB005 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> Recent metaheuristic developments have proved to be successful especially in cases where their fundamental concepts are complemented with pool-oriented approaches such as scatter search. These algorithms maintain a reference set of high quality solutions which are repeatedly used during the search in order to guarantee a fruitful balance between diversification and intensification. However, maintaining such a pool is not a trivial undertaking. The main problem is to find a balance between the attempt to collect a number of high quality solutions, which often results in similar solution properties, and the need to guarantee a certain degree of diversity in the pool. In the present study we highlight the advantage of pool-oriented design while focusing on controlled input and output operations. We analyze the main pool components for metaheuristics and present a review of the literature on relevant strategies in tabu search, scatter search, and path relinking. Additionally, we provide a synthesis of the literature that highlights the critical aspects of effective pool maintenance. <s> BIB006 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> In this paper we derive the probability distribution of trial points in the differential evolution (de) algorithm, in particular the probability distribution of points generated by mutation. We propose a point generation scheme that uses an approximation to this distribution. The scheme can dispense with the differential vector used in the mutation of de. We propose a de algorithm that replaces the differential based mutation scheme with a probability distribution based point generation scheme. We also propose a de algorithm that uses a probabilistic combination of the point generation by the probability distribution and the point generation by mutation. A numerical study is carried out using a set of 50 test problems, many of which are inspired by practical applications. Numerical results suggest that the new algorithms are superior to the original version both in terms of the number of function evaluations and cpu times. <s> BIB007 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> Although the Differential Evolution (DE) algorithm has been shown to be a simple yet powerful evolutionary algorithm for optimizing continuous functions, users are still faced with the problem of preliminary testing and hand-tuning of the evolutionary parameters prior to commencing the actual optimization process. As a solution, self-adaptation has been found to be highly beneficial in automatically and dynamically adjusting evolutionary parameters such as crossover rates and mutation rates. In this paper, we present a first attempt at self-adapting the population size parameter in addition to self-adapting crossover and mutation rates. Firstly, our main objective is to demonstrate the feasibility of self-adapting the population size parameter in DE. Using De Jong's F1–F5 benchmark test problems, we showed that DE with self-adaptive populations produced highly competitive results compared to a conventional DE algorithm with static populations. In addition to reducing the number of parameters used in DE, the proposed algorithm actually outperformed the conventional DE algorithm for one of the test problems. It was also found that that an absolute encoding methodology for self-adapting population size in DE produced results with greater optimization reliability compared to a relative encoding methodology. <s> BIB008 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> The particle swarm algorithm uses principles derived from social psychology to find optimal points in a search space. The present paper decomposes and reinterprets the particle swarm in order to discover new ways of implementing the algorithm. Some essential characteristics of the method are illuminated, and some inessential features are discarded. Various new forms are tested and found to perform well on a suite of test functions. In particular, it is shown that the traditional trajectory formulas can be replaced with random number generators sampling from various symmetrical probability distributions. The excellent performance of these new versions demonstrates that the strength of the algorithm is in the interactions of the particles, rather than in their behavior as individuals. <s> BIB009 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> This paper studies the efficiency of a recently defined population-based direct global optimization method called Differential Evolution with self-adaptive control parameters. The original version uses fixed population size but a method for gradually reducing population size is proposed in this paper. It improves the efficiency and robustness of the algorithm and can be applied to any variant of a Differential Evolution algorithm. The proposed modification is tested on commonly used benchmark problems for unconstrained optimization and compared with other optimization methods such as Evolutionary Algorithms and Evolution Strategies. <s> BIB010 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> This paper proposes an effective particle swarm optimization (PSO)-based memetic algorithm (MA) for the permutation flow shop scheduling problem (PFSSP) with the objective to minimize the maximum completion time, which is a typical non-deterministic polynomial-time (NP) hard combinatorial optimization problem. In the proposed PSO-based MA (PSOMA), both PSO-based searching operators and some special local searching operators are designed to balance the exploration and exploitation abilities. In particular, the PSOMA applies the evolutionary searching mechanism of PSO, which is characterized by individual improvement, population cooperation, and competition to effectively perform exploration. On the other hand, the PSOMA utilizes several adaptive local searches to perform exploitation. First, to make PSO suitable for solving PFSSP, a ranked-order value rule based on random key representation is presented to convert the continuous position values of particles to job permutations. Second, to generate an initial swarm with certain quality and diversity, the famous Nawaz-Enscore-Ham (NEH) heuristic is incorporated into the initialization of population. Third, to balance the exploration and exploitation abilities, after the standard PSO-based searching operation, a new local search technique named NEH_1 insertion is probabilistically applied to some good particles selected by using a roulette wheel mechanism with a specified probability. Fourth, to enrich the searching behaviors and to avoid premature convergence, a simulated annealing (SA)-based local search with multiple different neighborhoods is designed and incorporated into the PSOMA. Meanwhile, an effective adaptive meta-Lamarckian learning strategy is employed to decide which neighborhood to be used in SA-based local search. Finally, to further enhance the exploitation ability, a pairwise-based local search is applied after the SA-based search. Simulation results based on benchmarks demonstrate the effectiveness of the PSOMA. Additionally, the effects of some parameters on optimization performances are also discussed <s> BIB011 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> Combinatorial optimization problems arise in many applications such as task assignment, facility location, and elevator scheduling. A wide variety of population-based solution methods have been developed, either instance-based (e.g., genetic algorithm (GA) and particle swarm optimization (PSO)) or model-based (e.g., ant colony optimization(ACO) and estimation of distribution algorithms (EDAs)). Their various mechanisms make it difficult to analyze and compare these methods and to extend the advancement in one method to another. To this end, a unified optimization framework towards representing these seemingly different methods is established as iteratively sampling and updating of a population distribution. This framework is then innovatively instantiated with PSO from the instance-based category and EDA from the model-based category. Finally, the possible use and the finite time performance analysis of the unified framework are discussed. <s> BIB012 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> Recently, various multiobjective particle swarm optimization (MOPSO) algorithms have been developed to efficiently and effectively solve multiobjective optimization problems. How ever, the existing MOPSO designs generally adopt a notion to "estimate" a fixed population size sufficiently to explore the search space without incurring excessive computational complexity. To address the issue, this paper proposes the integration of a dynamic population strategy within the multiple-swarm MOPSO. The proposed algorithm is named dynamic population multiple-swarm MOPSO. An additional feature, adaptive local archives, is designed to improve the diversity within each swarm. Performance metrics and benchmark test functions are used to examine the performance of the proposed algorithm compared with that of five selected MOPSOs and two selected multiobjective evolutionary algorithms. In addition, the computational cost of the proposed algorithm is quantified and compared with that of the selected MOPSOs. The proposed algorithm shows competitive results with improved diversity and convergence and demands less computational cost. <s> BIB013 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> We propose a crossover-based adaptive local search (LS) operation for enhancing the performance of standard differential evolution (DE) algorithm. Incorporating LS heuristics is often very useful in designing an effective evolutionary algorithm for global optimization. However, determining a single LS length that can serve for a wide range of problems is a critical issue. We present a LS technique to solve this problem by adaptively adjusting the length of the search, using a hill-climbing heuristic. The emphasis of this paper is to demonstrate how this LS scheme can improve the performance of DE. Experimenting with a wide range of benchmark functions, we show that the proposed new version of DE, with the adaptive LS, performs better, or at least comparably, to classic DE algorithm. Performance comparisons with other LS heuristics and with some other well-known evolutionary algorithms from literature are also presented. <s> BIB014 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> This paper proposes a hybrid algorithm based on particle swarm optimization (PSO) for a multiobjective permutation flow shop scheduling problem, which is a typical NP-hard combinatorial optimization problem with strong engineering backgrounds. Not only does the proposed multiobjective algorithm (named MOPSO) apply the parallel evolution mechanism of PSO characterized by individual improvement, population cooperation, and competition to effectively perform exploration but it also utilizes several adaptive local search methods to perform exploitation. First, to make PSO suitable for solving scheduling problems, a ranked-order value (ROV) rule based on a random key technique to convert the continuous position values of particles to job permutations is presented. Second, a multiobjective local search based on the Nawaz-Enscore-Ham heuristic is applied to good solutions with a specified probability to enhance the exploitation ability. Third, to enrich the searching behavior and to avoid premature convergence, a multiobjective local search based on simulated annealing with multiple different neighborhoods is designed, and an adaptive meta-Lamarckian learning strategy is employed to decide which neighborhood will be used. Due to the fusion of multiple different searching operations, good solutions approximating the real Pareto front can be obtained. In addition, MOPSO adopts a random weighted linear sum function to aggregate multiple objectives to a single one for solution evaluation and for guiding the evolution process in the multiobjective sense. Due to the randomness of weights, searching direction can be enriched, and solutions with good diversity can be obtained. Simulation results and comparisons based on a variety of instances demonstrate the effectiveness, efficiency, and robustness of the proposed hybrid algorithm. <s> BIB015 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> Several theoretical analyses of the dynamics of particle swarms have been offered in the literature over the last decade. Virtually all rely on substantial simplifications, often including the assumption that the particles are deterministic. This has prevented the exact characterization of the sampling distribution of the particle swarm optimizer (PSO). In this paper we introduce a novel method that allows us to exactly determine all the characteristics of a PSO sampling distribution and explain how it changes over any number of generations, in the presence stochasticity. The only assumption we make is stagnation, i.e., we study the sampling distribution produced by particles in search for a better personal best. We apply the analysis to the PSO with inertia weight, but the analysis is also valid for the PSO with constriction and other forms of PSO. <s> BIB016 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> The study and research of evolutionary algorithms (EAs) is getting great attention in recent years. Although EAs have earned extensive acceptance through numerous successful applications in many fields, the problem of finding the best combination of evolutionary parameters especially for population size that need the manual settings by the user is still unresolved. In this paper, our system is focusing on differential evolution (DE) and its control parameters. To overcome the problem, two new systems were carried out for the self-adaptive population size to test two different methodologies (absolute encoding and relative encoding) in DE and compared their performances against the original DE. Fifty runs are conducted for every 20 well-known benchmark problems to test on every proposed algorithm in this paper to achieve the function optimization without explicit parameter tuning in DE. The empirical testing results showed that DE with self-adaptive population size using relative encoding performed well in terms of the average performance as well as stability compared to absolute encoding version as well as the original DE. <s> BIB017 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> The particle swarm optimizer (PSO) is a population-based optimization technique that can be applied to a wide range of problems. This paper presents a variation on the traditional PSO algorithm, called the efficient population utilization strategy for PSO (EPUS-PSO), adopting a population manager to significantly improve the efficiency of PSO. This is achieved by using variable particles in swarms to enhance the searching ability and drive particles more efficiently. Moreover, sharing principals are constructed to stop particles from falling into the local minimum and make the global optimal solution easier found by particles. Experiments were conducted on unimodal and multimodal test functions such as quadric, griewanks, rastrigin, ackley, and weierstrass, with and without coordinate rotation. The results show good performance of the EPUS-PSO in solving most benchmark problems as compared to other recent variants of the PSO. <s> BIB018 </s> Hybridizing Differential Evolution and Particle Swarm Optimization to Design Powerful Optimizers: A Review and Taxonomy <s> C. Unified Views on Particle Swarm Optimization and Differential Evolution <s> This paper proposes an effective hybrid algorithm based on differential evolution (DE), namely HDE, to solve multi-objective permutation flow shop scheduling problem (MPFSSP) with limited buffers between consecutive machines, which is a typical NP-hard combinatorial optimization problem with strong engineering background. Firstly, to make DE suitable for solving scheduling problems, a largest-order-value (LOV) rule is presented to convert the continuous values of individuals in DE to job permutations. Secondly, after the DE-based exploration, an efficient local search, which is designed based on the landscape of MPFSSP with limited buffers, is applied to emphasize exploitation. Thus, not only does the HDE apply the parallel evolution mechanism of DE to perform effective exploration (global search) in the whole solution space, but it also adopts problem-dependent local search to perform thorough exploitation (local search) in the promising sub-regions. In addition, the concept of Pareto dominance is used to handle the updating of solutions in sense of multi-objective optimization. Moreover, the convergence property of HDE is analyzed by using the theory of finite Markov chain. Finally, simulations and comparisons based on benchmarks demonstrate the effectiveness and efficiency of the proposed HDE. <s> BIB019
A unified view is advantageous to understand the similarities and differences of diverse optimizers and identify the fundamental ingredients for hybridization. From the introduction given earlier, it is easy to see that both DE and PSO are POs, manipulating multiple solutions in parallel. Sun et al. pointed out that the essence of population-based optimizers is to construct the sampling probability distribution from which the next population is generated BIB012 . Regarding the sampling distribution of DE and PSO, several scholars have conducted some pioneering research BIB007 BIB016 BIB003 BIB009 . Ali et al. derived the probability distribution of trial points in DE and proposed a point generation scheme to approximate the distribution BIB007 . Poli analyzed the sampling distribution of a classical PSO with the assumption of stagnation and used it to explain the search behavior of PSO BIB016 . Particularly, Kennedy proposed a simplified PSO version [named barebones PSO (BBPSO)] with an attempt to more concisely reflect the essence of the PSO paradigm, i.e., the social interaction of individuals BIB003 . In this compact PSO, the velocity formula is eliminated, while the sampling distribution of classical PSO is retained by the employment of a Guassian sampling approach. Kennedy suggests the normal distribution x for the Guassian sampling on each dimension BIB003 . It is evident that the direct sampling approach still encompasses the collaborative interaction of population members BIB009 . Omran et al. borrowed the idea of Kennedy's BBPSO to design a self-adaptive barebones DE based on DE/rand/1/bin  . The base vector and individual difference of DE/rand/1/bin are taken as the mean and standard deviation of the Gaussian sampling distribution for the barebones DE, respectively. In terms of the sampling approach, there is a big difference between DE and PSO. The sampling of PSO integrates individual cognition (pbest) and social collaboration (nbest). In contrast, individual experience (pbest) is seemingly not utilized in DE to guide its sampling behavior. DE relies on differential mutation and crossover to generate trial points by means of the usage of the difference of randomly selected individuals to perturb base vectors. However, it is worth noting that the one-to-one tournament selection that is employed by DE ensures any individual to be its own up-to-date pbest. Stated another way, at any time k, pbest (This is usually not the case for PSO.) In view of this fact, we can deem the sampling of DE as a combination of pbest and perturbed base vectors [see (1) and (2)]. Particularly, in DE/best/2/bin, the base vector is the so-far-best individual (i.e., gbest). Therefore, this DE variant shares some similarity with the Gbest PSO in terms of the sampling approach. In BIB005 , Xie and Zhang proposed a swarm algorithm framework (SWAF) that is realized by agent-based modeling, regarding each individual (agent) as a barebones cognitive architecture, which gains knowledge by the appropriate deployment of a set of simple rules. In fact, the SWAF provides us a template from the perspective of cognition to comprehend the similarities and differences of diverse population-based metaheuristics that include PSO and DE. In the SWAF, PSO, and DE, to employ different generate-and-test rules, are both regarded as socially biased individual learning heuristics BIB005 . On the implementation of diverse metaheuristics, Taillard et al. set forth a unifying view from the perspective of adaptive memory-based programming BIB002 . Although the concept of memory is usually regarded as a kernel in tabu search  and it is usually not stressed in the DE and PSO literature, both DE and PSO have to maintain a memory to record personal best information. Calégari et al. proposed a taxonomy of EAs, taking into account multiple classification criteria, such as population sizing strategy and population structure BIB001 . Both DE and PSO can adopt a static (constant) or dynamic population size BIB008 BIB017 BIB010 BIB018 BIB013 ; howbeit, they differ in terms of population structure as DE originally does not differentiate population topologies. Greistorfer and Voβ proposed a "pool" template to cover diverse classes of metaheuristics BIB004 , BIB006 . The template uses the metaphor of "pool" to describe the set of candidate solutions that are possibly chosen as ingredients for subsequent recombination methods or as start solutions for improvement methods (IMs). In fact, the pool is a concept that is equivalent to the memory, which Taillard et al. emphasized in BIB002 . Various metaheuristics can be differentiated in terms of IM, solution combination method (SCM), and the use and update of the pool. We can also comprehend the similarities and differences between DE and PSO according to this template. First, as mentioned earlier, neither classical DE nor classical PSO employs IMs though there exist several advanced variants of DE and PSO, which incorporate LS techniques as an improvement strategy BIB014 BIB019  BIB011 BIB015 . Second, DE takes differential mutation and crossover as its SCM, while the SCM of PSO is a random combination of pbest and nbest. As for the pool, both DE and PSO keep only a pool of pbest information, which contains nbest and gbest information, that is to say, any visited solution that is inferior to pbest will be discarded and not used in subsequent samplings. The use and update of the pool here implies a somewhat greedy selection scheme and a higher selection pressure. Purely relying on the pbest information to guide the search may result in loss of valuable information that is contained in other samplings and miss some opportunities to achieve a more efficient exploration or exploitation. An in-depth utilization of the search history that includes those inferior solutions provides us a viable way to improve the performance of optimizers. To summarize, DE and PSO are similar in terms of the basic features of POs. They differ mainly in terms of sampling approach. DE depends on a difference-based perturbation and crossover to achieve its sampling, while PSO follows the paradigm to combine individual cognition and populationtopology-dependent social learning.