Swarm robotics: a review from the swarm engineering perspective <s> Evolutionary robotics Evolutionary robotics <s> Time underlies many interesting human behaviors. Thus, the question of how to represent time in connectionist models is very important. One approach is to represent time implicitly by its effects on processing rather than explicitly (as in a spatial representation). The current report develops a proposal along these lines first described by Jordan (1986) which involves the use of recurrent links in order to provide networks with a dynamic memory. In this approach, hidden unit patterns are fed back to themselves; the internal representations which develop thus reflect task demands in the context of prior internal states. A set of simulations is reported which range from relatively simple problems (temporal version of XOR) to discovering syntactic/semantic features for words. The networks are able to learn interesting internal representations which incorporate task demands with memory demands; indeed, in this approach the notion of memory is inextricably bound up with task processing. These representations reveal a rich structure, which allows them to be highly context-dependent while also expressing generalizations across classes of items. These representations suggest a method for representing lexical categories and the type/token distinction. <s> BIB001 </s> Swarm robotics: a review from the swarm engineering perspective <s> Evolutionary robotics Evolutionary robotics <s> We would like the behavior of the artificial agents that we construct to be as well-adapted to their environments as natural animals are to theirs. Unfortunately, designing controllers with these properties is a very difficult task. In this article, we demonstrate that continuous-time recurrent neural networks are a viable mechanism for adaptive agent control and that the genetic algorithm can be used to evolve effective neural controllers. A significant advantage of this approach is that one need specify only a measure of an agent's overall performance rather than the precise motor output trajectories by which it is achieved. By manipulating the performance evaluation, one can place selective pressure on the development of controllers with desired properties. Several novel controllers have been evolved, including a chemotaxis controller that switches between different strategies depending on environmental conditions, and a locomotion controller that takes advantage of sensory feedback if available but th... <s> BIB002 </s> Swarm robotics: a review from the swarm engineering perspective <s> Evolutionary robotics Evolutionary robotics <s> This paper discusses the feasibility of applying evolutionary methods to automatically generating controllers for physical mobile robots. We overview the state-of-the-art in the field, describe some of the main approaches, discuss the key challenges, unanswered problems, and some promising directions. <s> BIB003 </s> Swarm robotics: a review from the swarm engineering perspective <s> Evolutionary robotics Evolutionary robotics <s> In this paper, the effects of adaptive robotic behavior via learning momentum in the context of a robotic team are studied. Learning momentum is a variation on parametric adjustment methods that has previously been successfully applied to enhance individual robot performance. In particular, we now assess, via simulation, the potential advantages of a team of robots using this capability to alter behavioral parameters when compared to a similar team of robots with static parameters. <s> BIB004 </s> Swarm robotics: a review from the swarm engineering perspective <s> Evolutionary robotics Evolutionary robotics <s> This paper addresses qualitative and quantitative diversity and specialization issues in the framework of self-organizing, distributed, artificial systems. Both diversity and specialization are obtained via distributed learning from initially homogeneous swarms. While measuring diversity essentially quantifies differences among the individuals, assessing the degree of specialization implies correlation between the swarm's heterogeneity with its overall performance. Starting from the stick-pulling experiment in collective robotics, a task that requires the collaboration of two robots, we abstract and generalize in simulation the task constraints to k robots collaborating sequentially or in parallel. We investigate quantitatively the influence of task constraints and types of reinforcement signals on performance, diversity, and specialization in these collaborative experiments. Results show that, though diversity is not explicitly rewarded in our learning algorithm, even in scenarios without explicit communication among agents the swarm becomes specialized after learning. The degrees of both diversity and specialization are affected strongly by environmental conditions and task constraints. While the specialization measure reveals characteristics related to performance and learning in a clearer way than diversity does, the latter measure appears to be less sensitive to different noise conditions and learning parameters. <s> BIB005 </s> Swarm robotics: a review from the swarm engineering perspective <s> Evolutionary robotics Evolutionary robotics <s> An important goal of collective robotics is the design of control systems that allow groups of robots to accomplish common tasks by coordinating without a centralized control. In this paper, we study how a group of physically assembled robots can display coherent behavior on the basis of a simple neural controller that has access only to local sensory information. This controller is synthesized through artificial evolution in a simulated environment in order to let the robots display coordinated-motion behaviors. The evolved controller proves to be robust enough to allow a smooth transfer from simulated to real robots. Additionally, it generalizes to new experimental conditions, such as different sizes/shapes of the group and/or different connection mechanisms. In all these conditions the performance of the neural controller in real robots is comparable to the one obtained in simulation <s> BIB006 </s> Swarm robotics: a review from the swarm engineering perspective <s> Evolutionary robotics Evolutionary robotics <s> Traditional approaches to designing multi-agent systems are offline, in simulation, and assume the presence of a global observer. Artificial Physics (AP) or physicomimetics can be used to self-organize swarms of mobile robots into formations that move towards a goal. Using an offline approach, we extend the AP framework to moving formations through obstacle fields. We provide important metrics of performance that allow us to (a) compare the utility of different generalized force laws in the artificial physics framework, (b) examine trade-offs between different metrics, and (c) provide a detailed method of comparison for future researchers in this area. ::: In the online, real world, a global observer may be absent, performance feedback may be delayed or perturbed by noise, agents may only interact with their local neighbors, and only a subset of agents may experience any form of performance feedback. Under these constraints, designing multi-agent systems is difficult. We present a novel approach called "Distributed Agent Evolution with Dynamic Adaptation to Local Unexpected Scenarios'' or DAEDALUS to address these issues, by mimicking more closely the actual dynamics of populations of agents moving and interacting in a (task) environment. ::: This thesis merges DAEDALUS and AP by using obstacle avoidance as a case study to illustrate the feasibility of DAEDALUS when the environment changes. We present empirical and practical results that address (a) offline vs. online learning, (b) obstructed perception, (c) homogeneous vs. heterogeneous agent cooperation, and (d) implementation of obstacle avoidance with real robots. <s> BIB007 </s> Swarm robotics: a review from the swarm engineering perspective <s> Evolutionary robotics Evolutionary robotics <s> Designing effective behavioral controllers for mobile robots can be difficult and tedious; this process can be circumvented by using unsupervised learning techniques which allow robots to evolve their own controllers in an automated fashion. In multi-robot systems, robots learning in parallel can share information to dramatically increase the evolutionary rate. However, manufacturing variations in robotic sensors may result in perceptual differences between robots, which could impact the learning process. In this paper, we explore how varying sensor offsets and scaling factors affects parallel swarm-robotic learning of obstacle avoidance behavior using both Genetic Algorithms and Particle Swarm Optimization. We also observe the diversity of robotic controllers throughout the learning process in an attempt to better understand the evolutionary process. <s> BIB008 </s> Swarm robotics: a review from the swarm engineering perspective <s> Evolutionary robotics Evolutionary robotics <s> Group transport is performed in many natural systems and has become a canonical task for studying cooperation in robotics. We simulate a system of simple, insect-like robots that can move autonomously and grasp objects as well as each other. We use artificial evolution to produce solitary transport and group transport behaviors. We show that robots, even though not aware of each other, can be effective in group transport. Group transport can even be performed by robots that behave as in solitary transport. Still, robots engaged in group transport can benefit from behaving differently from robots engaged in solitary transport. The best group transport behaviors yielded by half of the evolutions let robots organize into self-assembled structures. This provides evidence that self-assembly can provide adaptive value to individuals that compete in an artificial evolution based on task performance. We conclude the article by discussing potential implications for evolutionary biology and robotics. <s> BIB009 </s> Swarm robotics: a review from the swarm engineering perspective <s> Evolutionary robotics Evolutionary robotics <s> A well known problem in the design of the control system for a swarm of robots concerns the definition of suitable individual rules that result in the desired coordinated behaviour. A possible solution to this problem is given by the automatic synthesis of the individual controllers through evolutionary or learning processes. These processes offer the possibility to freely search the space of the possible solutions for a given task, under the guidance of a user-defined utility function. Nonetheless, there exist no general principles to follow in the definition of such a utility function in order to reward coordinated group behaviours. As a consequence, task dependent functions must be devised each time a new coordination problem is under study. In this paper, we propose the use of measures developed in Information Theory as task-independent, implicit utility functions. We present two experiments in which three robots are trained to produce generic coordinated behaviours. Each robot is provided with rich sensory and motor apparatus, which can be exploited to explore the environment and to communicate with other robots. We show how coordinated behaviours can be synthesised through a simple evolutionary process. The only criteria used to evaluate the performance of the robotic group is the estimate of mutual information between the motor states of the robots. <s> BIB010 </s> Swarm robotics: a review from the swarm engineering perspective <s> Evolutionary robotics Evolutionary robotics <s> Communication is of central importance in collective robotics, as it is integral to the switch from solitary to social behavior. In this article, we study emergent communication behaviors that are not predetermined by the experimenter, but are shaped by artificial evolution, together with the rest of the behavioral repertoire of the robots. In particular, we describe a set of experiments in which artificial evolution is used as a means to engineer robot neuro-controllers capable of guiding groups of robots in a categorization task by producing appropriate actions. The categorization is a result of how robots' sensory inputs unfold in time, and, more specifically, of the integration over time of sensory input. In spite of the absence of explicit selective pressure (coded into the fitness function), which would favor signaling over non-signaling groups, communicative behavior emerges. Post-evaluation analyses illustrate the adaptive function of the evolved signals and show that these signals are tightly linked to the behavioral repertoire of the agents. Signals evolve because communication enhances group performance, revealing a “hidden” benefit for social behavior. This benefit is related to obtaining robust and fast decision-making mechanisms. More generally, we show how processes requiring the categorization of noisy dynamical information might be improved by social interactions mediated by communication. In a further series of experiments, we successfully download evolved controllers onto real s-bots. We discuss the challenges involved in porting neuro-controllers displaying time-based decision-making processes onto real robots. Finally, the beneficial effect of communication is shown to transfer to the case of a real robot, and the robustness of the behavior against inter-robot differences is discussed. <s> BIB011 </s> Swarm robotics: a review from the swarm engineering perspective <s> Evolutionary robotics Evolutionary robotics <s> Many collaborative multi-robot application domains have limited areas of operation that cause spatial conflicts between robotic teammates. These spatial conflicts can cause the team's productivity to drop with the addition of robots. This phenomenon is impacted by the coordination methods used by the team-members, as different coordination methods yield radically different productivity results. However, selecting the best coordination method to be used by teammates is a formidable task. This paper presents techniques for creating adaptive coordination methods to address this challenge. We first present a combined coordination cost measure, CCC, to quantify the cost of group interactions. Our measure is useful for facilitating comparison between coordination methods, even when multiple cost factors are considered. We consistently find that as CCC values grow, group productivity falls. Using the CCC, we create adaptive coordination techniques that are able to dynamically adjust the efforts spent on coordination to match the number of perceived coordination conflicts in a group. We present two adaptation heuristics that are completely distributed and require no communication between robots. Using these heuristics, robots independently estimate their combined coordination cost (CCC), adjust their coordination methods to minimize it, and increase group productivity. We use simulated robots to perform thousands of experiment trials to demonstrate the efficacy of our approach. We show that using adaptive coordination methods create a statistically significant improvement in productivity over static methods, regardless of the group size. <s> BIB012 </s> Swarm robotics: a review from the swarm engineering perspective <s> Evolutionary robotics Evolutionary robotics <s> In cooperative multiagent systems, agents interact to solve tasks. Global dynamics of multiagent teams result from local agent interactions, and are complex and difficult to predict. Evolutionary computation has proven a promising approach to the design of such teams. The majority of current studies use teams composed of agents with identical control rules (ldquogenetically homogeneous teamsrdquo) and select behavior at the team level (ldquoteam-level selectionrdquo). Here we extend current approaches to include four combinations of genetic team composition and level of selection. We compare the performance of genetically homogeneous teams evolved with individual-level selection, genetically homogeneous teams evolved with team-level selection, genetically heterogeneous teams evolved with individual-level selection, and genetically heterogeneous teams evolved with team-level selection. We use a simulated foraging task to show that the optimal combination depends on the amount of cooperation required by the task. Accordingly, we distinguish between three types of cooperative tasks and suggest guidelines for the optimal choice of genetic team composition and level of selection. <s> BIB013
Within swarm robotics, ER has been used in many proof-of-concept tasks in order to test the effectiveness of the method BIB006 BIB009 BIB010 or as a tool to answer some more fundamental scientific questions   BIB011 . In this review, we analyze ER from an engineering perspective, that is, we describe its strengths and weaknesses as a design method. The ER method can be described by the following process. At the beginning, a population of individual behaviors is generated at random. In each iteration, a number of experiments for each individual behavior is executed. The same individual behavior is used by all the robots in the experiment. In each experiment, a fitness function is used to evaluate the collective behavior of the swarm resulting from that individual behavior. At this point, a selection of the highest scoring individual behaviors are modified by genetic operators, such as cross-over and mutation, and used for the subsequent iterations. In the majority of the works, the group is homogeneous (all individual behaviors are the same) and the fitness evaluates the performance of the entire swarm. BIB013 introduced two taxonomies: one of the taxonomies distinguishes works according to how fitness is computed (individual-level vs swarm level), whereas the other one according to how the swarm is composed (homogeneous vs heterogeneous). In ER, the individual behavior can be represented in many ways, such as finite state machines or virtual force functions BIB007 . Typically, the evolutionary method is used to find the parameters of an artificial neural network (henceforth NN). Although several types of NN exist in the literature, they can roughly be categorized in two main classes: feed-forward NN  and recurrent NN BIB002 BIB001 ). Feed-forward NNs are used for individual behaviors that require no memory of previous observations and actions. Conversely, recurrent NNs are used for individual behaviors that require a memory of previously seen input patterns. ER with recurrent neural networks has been extensively studied in swarm robotics by  . RL and ER have many common points. In fact, the same problems identified by us in the application of RL to swarm robotics apply also to ER. Other problems, many of which were identified by BIB003 , are instead related only to ER: (i) Evolution is a computationally intensive process that does not give any guarantees on its convergence to a solution; (ii) neural networks are black-box and it is often very difficult to understand their behavior; (iii) from an engineering point of view, the complexity of behaviors currently synthesized through artificial evolution is relatively low and the same results may often be achieved by designing the behavior by hand. Other learning and automatic design methods In this section, we outline other works on automatic design that can be placed neither within the RL nor within the ER literature. In all these works, the authors design an individual behavior with parameters to be found, and they use some algorithm to automatically find (or learn) these parameters on-line, that is, while the robots are interacting with the environment.  proposed ALLIANCE, a multi-robot architecture that focuses on the achievement of fault tolerant, robust, and adaptive task allocation in a team of robots. In her work, the author added the possibility to perform on-line learning of the parameters of ALLIANCE. BIB004 extended their learning momentum framework to multi-robot systems. With learning momentum, it is possible to learn on-line the behavioral parameters according to the situations robots are facing. These parameters are the weight assigned to vectors produced by different, virtual physics-based, sub-behaviors. BIB005 proposed an algorithm that enables on-line learning of some parameters of the robot behaviors in order to achieve diversity and specialization in a swarm of robots. The learning algorithm is specifically thought for their application, which consists of a stick pulling task. BIB007 used the virtual-physics-based design method combined with evolutionary computation. He used genetic algorithms to learn off-line the parameters of the Lennard-Jones potential function (see Sect. 2.1.1) in a navigation with obstacle avoidance task. BIB012 studied the problem of how to obtain an adaptive coordination behavior in a multi-robot domain. Their first contribution has been to propose a method that enables each robot to estimate the coordination cost over time. An example of such cost is the one needed to reduce interference. They then proposed a learning algorithm that is able to produce a behavior mapping the current estimate of the coordination cost to the coordination method to be used. Finally, BIB008 compared the PSO (particle swarm optimization) algorithm against a genetic algorithm for on-line learning parameters for a swarm of robots performing obstacle avoidance. They also defined metrics to measure diversity and specialization, and concluded that PSO is able to achieve a higher degree of diversity in the swarm.