A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> Abstract This paper deals with the decision rules of a tree classifier for performing the classification at each nonterminal node, under the assumption of complete probabilistic information. For given tree structure and feature subsets to be used, the optimal decision rules (strategy) are derived which minimize the overall probability of misclassification. The primary result is illustrated by an example. <s> BIB001 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> Abstract Two different Artificial Neural Network (ANN) classifiers have been compared with a traditional closest-mean-classifier, a VQ and a Kohonen-network with respect to classification of vowels extracted from continuous speech. The first ANN-classifier is a standard three-layer perceptron network and the second ANN-classifier uses the response from a 16 × 16 Kohonen map as input to a two layer perceptron network. The result shows that the best performance is achieved with the two ANN-classifiers and indicates that a Kohonen map does not deteriorate the information presented to the second layer in the network and hence can be used instead of a first hidden layer. <s> BIB002 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> Evolutionary programming (EP) has been applied with success to many numerical and combinatorial optimization problems in recent years. EP has rather slow convergence rates, however, on some function optimization problems. In the paper, a "fast EP" (FEP) is proposed which uses a Cauchy instead of Gaussian mutation as the primary search operator. The relationship between FEP and classical EP (CEP) is similar to that between fast simulated annealing and the classical version. Both analytical and empirical studies have been carried out to evaluate the performance of FEP and CEP for different function optimization problems. The paper shows that FEP is very good at search in a large neighborhood while CEP is better at search in a small local neighborhood. For a suite of 23 benchmark problems, FEP performs much better than CEP for multimodal functions with many local minima while being comparable to CEP in performance for unimodal and multimodal functions with only a few local minima. The paper also shows the relationship between the search step size and the probability of finding a global optimum and thus explains why FEP performs better than CEP on some functions but not on others. In addition, the importance of the neighborhood size and its relationship to the probability of finding a near-optimum is investigated. Based on these analyses, an improved FEP (IFEP) is proposed and tested empirically. This technique mixes different search operators (mutations). The experimental results show that IFEP performs better than or as well as the better of FEP and CEP for most benchmark problems tested. <s> BIB003 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> Particle Swarm Optimization (PSO) methods for dynamic function optimization are studied in this paper. We compare dynamic variants of standard PSO and Hierarchical PSO (H-PSO) on different dynamic benchmark functions. Moreover, a new type of hierarchical PSO, called Partitioned H-PSO (PH-PSO), is proposed. In this algorithm the hierarchy is partitioned into several sub-swarms for a limited number of generations after a change occurred. Different methods for determining the time when to rejoin the hierarchy and how to handle the topmost sub-swarm are discussed. The test results show that H-PSO performs significantly better than PSO on all test functions and that the PH-PSO algorithms often perform best on multimodal functions where changes are not too severe. <s> BIB004 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> Over the recent years, several studies have been carried out by the researchers to describe a general, flexible and powerful design method based on modern heuristic optimisation algorithms for infinite impulse response (IIR) digital filters since these algorithms have the ability of finding global optimal solution in a nonlinear search space. One of the modern heuristic algorithms is the artificial immune algorithm which implements a learning technique inspired by human immune system. However, the immune system has not attracted the same kind of interest from researchers as other heuristic algorithms. In this work, an artificial immune algorithm is described and applied to the design of IIR filters, and its performance is compared to that of genetic and touring ant colony optimisation algorithms. <s> BIB005 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> In this paper a new algorithm for edge detection using ant colony search is proposed. The problem is represented by a directed graph in which nodes are the pixels of an image. To adapt the problem, some modifications on original ant colony search algorithm (ACSA) are applied. A large number of experiments are employed to determine suitable algorithm parameters. We drive an experimental relationship between the size of the image to be analyzed and algorithm parameters. Several experiments are made and the results suggest the effectiveness of the proposed algorithm. <s> BIB006 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> Swarm intelligence is a research branch that models the population of interacting agents or swarms that are able to self-organize. An ant colony, a flock of birds or an immune system is a typical example of a swarm system. Bees' swarming around their hive is another example of swarm intelligence. Artificial Bee Colony (ABC) Algorithm is an optimization algorithm based on the intelligent behaviour of honey bee swarm. In this work, ABC algorithm is used for optimizing multivariable functions and the results produced by ABC, Genetic Algorithm (GA), Particle Swarm Algorithm (PSO) and Particle Swarm Inspired Evolutionary Algorithm (PS-EA) have been compared. The results showed that ABC outperforms the other algorithms. <s> BIB007 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> Optimization in dynamic environments is important in real-world applications, which requires the optimization algorithms to be able to find and track the changing optimum efficiently over time. Among various algorithms for dynamic optimization, particle swarm optimization algorithms (PSOs) are attracting more and more attentions in recent years, due to their ability of keeping good balance between convergence and diversity maintenance. To tackle the challenges of dynamic optimization, several strategies have been proposed to enhance the performance of PSO, and have gained success on various dynamic optimization problems. But there still exist some issues in dynamic optimization which need to be studied carefully, i.e. the robustness of the algorithm to problems of various dynamic features. In this paper, a new multi-strategy ensemble particle swarm optimization (MEPSO) for dynamic optimization is proposed. In MEPSO, all particles are divided into two parts, denoted as part I and part II, respectively. Two new strategies, Gaussian local search and differential mutation, are introduced into these two parts, respectively. Experimental analyses reveal that the mechanisms used in part I can enhance the convergence ability of the algorithm, while mechanisms used in part II can extend the searching area of the particle population to avoid being trapped into the local optimum, and can enhance the ability of catching up with the changing optimum in dynamic environments. The whole algorithm has few parameters that need to be tuned, and all of them are not sensitive to problems. We compared MEPSO with other PSOs, including MQSO, PHPSO and Standard PSO with re-initialization, on moving peaks Benchmark and dynamic Rastrigin function. The experimental results show that MEPSO has pretty good performance on almost all testing problems adopted in this paper, and outperforms other algorithms when the dynamic environment is unimodal and changes severely, or has a great number of local optima as dynamic Rastrigin function does. <s> BIB008 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> In this paper, a tabu search based clustering approach called TS-Clustering is proposed to deal with the minimum sum-of-squares clustering problem. In the TS-Clustering algorithm, five improvement operations and three neighborhood modes are given. The improvement operation is used to enhance the clustering solution obtained in the process of iterations, and the neighborhood mode is used to create the neighborhood of tabu search. The superiority of the proposed method over some known clustering techniques is demonstrated for artificial and real life data sets. <s> BIB009 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> This paper proposes a novel tuning strategy for robust proportional-integral-derivative (PID) controllers based on the augmented Lagrangian particle swarm optimization (ALPSO). First, the problem of PID controller tuning satisfying multiple H ∞ performance criteria is considered, which is known to suffer from computational intractability and conservatism when any existing method is adopted. In order to give some remedy to such a design problem without using any complicated manipulations, the ALPSO based robust gain tuning scheme for PID controllers is introduced. It does not need any conservative assumption unlike the conventional methods, and often enables us to find the desired PID gains just by solving the constrained optimization problem in a straightforward way. However, it is difficult to guarantee its effectiveness in a theoretical way, because PSO is essentially a stochastic approach. Therefore, it is evaluated by several simulation examples, which demonstrate that the proposed approach works well to obtain PID controller parameters satisfying the multiple H ∞ performance criteria. <s> BIB010 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> This paper presents a particle swarm optimization (PSO) algorithm to solve the parameter estimation problem for nonlinear dynamic rational filters. For the modeling of the nonlinear rational filter, the unknown filter parameters are arranged in the form of a parameter vector which is called a particle in the terminology of PSO. The proposed PSO algorithm applies the velocity updating and position updating formulas to the population composed of many particles such that better particles are generated. Because the PSO algorithm manipulates the parameter vectors directly as real numbers rather than binary strings, implementing the PSO algorithm into the computer programs becomes fairly easy and straightforward. Finally, an illustrative example for the modeling of the nonlinear rational filter is provided to show the validity, as compared with the traditional genetic algorithm, of the proposed method. <s> BIB011 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> Decision tree classification is one of the most practical and effective methods which is used in inductive learning. Many different approaches, which are usually used for decision making and prediction, have been invented to construct decision tree classifiers. These approaches try to optimize parameters such as accuracy, speed of classification, size of constructed trees, learning speed, and the amount of used memory. There is a trade off between these parameters. That is to say that optimization of one may cause obstruction in the other, hence all existing approaches try to establish equilibrium. In this study, considering the effect of the whole data set on class assigning of any data, we propose a new approach to construct not perfectly accurate, but less complex trees in a short time, using small amount of memory. To achieve this purpose, a multi-step process has been used. We trace the training data set twice in any step, from the beginning to the end and vice versa, to extract the class pattern for attribute selection. Using the selected attribute, we make new branches in the tree. After making branches, the selected attribute and some records of training data set are deleted at the end of any step. This process continues alternatively in several steps for remaining data and attributes until the tree is completely constructed. In order to have an optimized tree the parameters which we use in this algorithm are optimized using genetic algorithms. In order to compare this new approach with previous ones we used some known data sets which have been used in different researches. This approach has been compared with others based on the classification accuracy and also the decision tree size. Experimental results show that it is efficient to use this approach particularly in cases of massive data sets, memory restrictions or short learning time. <s> BIB012 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> This paper presents an adaptation of a novel algorithm based on the foraging behavior of honey bees to solve constrained numerical optimization problems. The modifications focus on improving the way the feasible region is approached by using a new operator which allows the generation of search directions biased by the best solution so far. Furthermore, two dynamic tolerances applied in the constraint handling mechanism help the algorithm to the generation of feasible solutions. The approach is tested on a set of 24 benchmark problems and its behavior is compared against the original algorithm and with respect to some state-of-the-art algorithms. <s> BIB013 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> Abstract Artificial bee colony (ABC) algorithm invented recently by Karaboga is a biological-inspired optimization algorithm, which has been shown to be competitive with some conventional biological-inspired algorithms, such as genetic algorithm (GA), differential evolution (DE) and particle swarm optimization (PSO). However, there is still an insufficiency in ABC algorithm regarding its solution search equation, which is good at exploration but poor at exploitation. Inspired by PSO, we propose an improved ABC algorithm called gbest-guided ABC (GABC) algorithm by incorporating the information of global best (gbest) solution into the solution search equation to improve the exploitation. The experimental results tested on a set of numerical benchmark functions show that GABC algorithm can outperform ABC algorithm in most of the experiments. <s> BIB014 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> Data clustering is one of the most popular techniques in data mining. It is a method of grouping data into clusters, in which each cluster must have data of great similarity and high dissimilarity with other cluster data. The most popular clustering algorithm K-mean and other classical algorithms suffer from disadvantages of initial centroid selection, local optima, low convergence rate problem etc. Particle Swarm Optimization (PSO) is a population based globalized search algorithm that mimics the capability (cognitive and social behavior) of swarms. PSO produces better results in complicated and multi-peak problems. This paper presents a literature survey on the PSO application in data clustering. PSO variants are also described in this paper. An attempt is made to provide a guide for the researchers who are working in the area of PSO and data clustering. <s> BIB015 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> Developing an efficient classification method is a challenge task in many research domains, such as neural network (NN) classifiers, statistical classifiers and machine learning. This study focuses on NN classifiers, which are data-driven analytical techniques. This study presents a cerebellar model articulation controller NN (CMAC NN) classifier, which has the advantages of very fast learning, reasonable generalization ability and robust noise resistance. To increase the accuracies of training and generalization, the CMAC NN classifier is designed with multiple-input and multiple-output (MIMO) network topology. The performance of the proposed MIMO CMAC NN classifier is evaluated using PROBEN1 benchmark datasets (such as for diabetes, cancer and glass) taken from the UCI Machine Learning Repository. Numerical results indicate that the proposed CMAC NN classifier is efficient for tested datasets. Moreover, this study compares the experimental results of the CMAC NN classifier with those in the published literature, indicating that the CMAC NN classifier is superior to some published classifiers. Therefore, the CMAC NN classifier can be considered as an analytical tool for solving classification tasks, such as medical decision making. <s> BIB016 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> Artificial Bee Colony (ABC) algorithm which is one of the most recently introduced optimization algorithms, simulates the intelligent foraging behavior of a honey bee swarm. Clustering analysis, used in many disciplines and applications, is an important tool and a descriptive task seeking to identify homogeneous groups of objects based on the values of their attributes. In this work, ABC is used for data clustering on benchmark problems and the performance of ABC algorithm is compared with Particle Swarm Optimization (PSO) algorithm and other nine classification techniques from the literature. Thirteen of typical test data sets from the UCI Machine Learning Repository are used to demonstrate the results of the techniques. The simulation results indicate that ABC algorithm can efficiently be used for multivariate data clustering. <s> BIB017 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> Clustering is used to group data objects into sets of disjoint classes called clusters so that objects within the same class are highly similar to each other and dissimilar from the objects in other classes. K-harmonic means (KHM) is one of the most popular clustering techniques, and has been applied widely and works well in many fields. But this method usually runs into local optima easily. A hybrid data clustering algorithm based on an improved version of Gravitational Search Algorithm and KHM, called IGSAKHM, is proposed in this research. With merits of both algorithms, IGSAKHM not only helps the KHM clustering to escape from local optima but also overcomes the slow convergence speed of the IGSA. The proposed method is compared with some existing algorithms on seven data sets, and the obtained results indicate that IGSAKHM is superior to KHM and PSOKHM in most cases. <s> BIB018 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> Swarm intelligence is a research field that models the collective intelligence in swarms of insects or animals. Many algorithms that simulates these models have been proposed in order to solve a wide range of problems. The Artificial Bee Colony algorithm is one of the most recent swarm intelligence based algorithms which simulates the foraging behaviour of honey bee colonies. In this work, modified versions of the Artificial Bee Colony algorithm are introduced and applied for efficiently solving real-parameter optimization problems. <s> BIB019 </s> A Review on Gravitational Search Algorithm and its Applications to Data Clustering & Classification <s> INTRODUCTION <s> In recent years, heuristic algorithms have been successfully applied to solve clustering and classification problems. In this paper, gravitational search algorithm (GSA) which is one of the newest swarm based heuristic algorithms is used to provide a prototype classifier to face the classification of instances in multi-class data sets. The proposed method employs GSA as a global searcher to find the best positions of the representatives (prototypes). The proposed GSA-based classifier is used for data classification of some of the well-known benchmark sets. Its performance is compared with the artificial bee colony (ABC), the particle swarm optimization (PSO), and nine other classifiers from the literature. The experimental results of twelve data sets from UCI machine learning repository confirm that the GSA can successfully be applied as a classifier to classification problems. <s> BIB020
Nature has always been a continuous source of inspiration for researchers and scientists. A large number of algorithms have been developed based on the natural process of evolution, laws, swarms behavior etc. Nature inspired algorithms are the latest state of art algorithms & works well with optimization problems as well as other problems than the classical methods because classical methods are inflexible in nature. It has been proved by many researchers that nature inspired algorithms are convenient to solve complex computational problems such as to optimize objective functions BIB008 BIB003 , pattern recognition BIB009  , control functions BIB010  , image processing  BIB006 , filter modeling BIB016 BIB005 , clustering BIB009 , classification BIB017 etc. In last one and half decade several nature inspired algorithms have been developed such as Particle swarm optimization (PSO), Genetic Algorithm (GA), Simulated Annealing (SA), Ant colony optimization (ACO), Artificial Bee colony (ABC) optimization, Big Bang Big Crunch (BB-BC) etc. These algorithms show better results than classical methods in terms of accuracy, convergence computational time etc. Kirkpatrick et al. [12] proposed simulated annealing algorithm based on the annealing process of metals and applied SA to solve many combinatorial optimization problems but SA suffered from convergence problem and trapped in local minima  BIB004 . In 1995, Kennady et al.  developed an algorithm based on the swarm behavior of birds & fish schooling and named it Particle Swarm Optimization (PSO). This algorithm was applied to solve many optimizations such as function optimization  but this algorithm suffered with partial optimism and not works with scattering problems. In case of complex and complicated problems PSO algorithm suffered with optimal solution BIB011 . In 1975, John Holland  developed an algorithm based on the natural process of evaluation and called it genetic algorithm. Many optimization problems solved by with the help of genetic algorithm but it's suffered from parameter tuning problem due to complicated crossover & mutation operator and convergence speed  . ACO was developed by the Dorigo [20] in 1992 based on the behaviors of real world ants. This algorithm applied to solve many optimization problems such as TSP [21] , Graph Coloring  , Network Routing  etc. ACO algorithm is sometimes suffered from selection bias  and convergence   problem. Devis Karbooga et al. BIB007 in 2006 proposed ABC algorithm based on the behavior of hooney Bees i.e. capabilities of Bees to find the food source for numerical optimization problems. Later on ABC algorithm has widely used to solve other problem such as multi-dimensional numeric problems  , real-parameter optimization BIB019  , constrained numerical optimization problems BIB013 etc. Dervis Karaboga & Beyza Gorkemli et al. BIB006 provided comprehensive description about the ABC algorithm and its application in various fields as well as hybridization of ABC & its variants. There are some drawbacks of ABC algorithm such as it requires large number of objective function evaluations  , handling complex multimodal functions, functions with narrow curve valley, convergence slow down due to stochastic nature  and weak exploitation feature BIB014 . Osman K. Erol et al.  Copyright © 2014 MECS I.J. Intelligent Systems and Applications, 2014, 06,  BIB018 BIB001 BIB012 BIB002 BIB015  BIB020  [88] [89] [90] [91] [92] [93] in 2006 proposed a new algorithm Big Bang Big Crunch (BB-BC) based on the big bang theory (Theory of Evolution of Universe) for optimization problems. BB-BC algorithm is used to solve large numbers of optimization problems such as multi modal optimization problem  , multi-objective optimization problem [36] , clustering  etc. A brief description about the nature inspired algorithms has been given above that are used to solve the optimization problems but till date there does not exists any algorithm that can solve the entire optimization problems exactly. The gravitational search algorithm is the latest nature inspired algorithm proposed by E. Rashedi [38] to solve the optimization problems based on the Law of gravity. Many researchers has applied the gravitational search algorithm on large numbers of problems because it requires only two parameters and having ability to find near global optimum solution and provides better results as compare to other nature inspired algorithms. Figure 1 depicts the applications of the gravitational search algorithm in various domains. The objective of this paper is to provide comprehensive survey for the researchers which are doing research in the field of optimizing problems (gravitational search algorithm), swarm intelligence and nature inspired algorithms. The first objective of this paper is to find out the number of algorithms that are derived from the original GSA to solve the optimization problems as well as to find out the application of gravitational search algorithm in various domains such as pattern recognition, power system, data mining, classification & clustering and how many algorithms are proposed by using gravitational search algorithm plus other nature inspired techniques such as GA, PSO, SA etc. (Hybrid GSA). The second objective of this paper is to find out the significance of Gravitational algorithm in clustering and classification problems. In present time, large number of algorithms based on the behavior of swarms & natural processes are used to solve these problems and this paper also provides the comparisons of GSA with swarm based algorithms for clustering and classification problems. The rest of paper is organized as follows. Section II gives the detail about GSA. Section III describes variants and modifications in GSA. The hybridization of GSA with other techniques is discussed in section IV. Section V and VI focus the application of GSA algorithm in clustering and classification. These sections are followed by conclusion of paper.