Division of labor has been studied separately from a proximate self-organization and an ultimate evolutionary perspective. We aim to bring together these two perspectives. So far this has been done by choosing a behavioral mechanism a priori and considering the evolution of the properties of this mechanism. Here we use artificial neural networks to allow for a more open architecture. We study whether emergent division of labor can evolve in two different network architectures; a simple feedforward network, and a more complex network that includes the possibility of self-feedback from previous experiences. We focus on two aspects of division of labor; worker specialization and the ratio of work performed for each task. Colony fitness is maximized by both reducing idleness and achieving a predefined optimal work ratio. Our results indicate that architectural constraints play an important role for the outcome of evolution. With the simplest network, only genetically determined specialization is possible. This imposes several limitations on worker specialization. Moreover, in order to minimize idleness, networks evolve a biased work ratio, even when an unbiased work ratio would be optimal. By adding self-feedback to the network we increase the network' s flexibility and worker specialization evolves under a wider parameter range. Optimal work ratios are more easily achieved with the self-feedback network, but still provide a challenge when combined with worker specialization.
Division of labor is ubiquitous in nature. The major evolutionary transitions, such as the separation of germ and soma and the transition from prokaryotes to eukaryotes, were accompanied by an increase in division of labor [1]. The transition from solitary to eusocial in insects encompasses the evolution of a reproductive caste and a sterile worker caste. Furthermore, division of labor among sterile workers also evolved, in which different groups of workers specialize in different functions, such as foraging and brood care [2]. Colony growth and survival is strongly dependent on the coordinated interaction of a large number of workers. This non-reproductive division of labor is therefore often considered a major determinant of the ecological success of eusocial insects and will be the focus of the work presented here. Empirical evidence suggests that eusociality has evolved in associations of close kin [3], [4]. Variation in behavioral tendencies can be found in forced associations of non-social individuals, leading to incipient forms of division of labor [5], [6]. Undoubtedly, a source of variation is key to generating consistent inter-individual differences and task specialization [7]. The questions that arise are how and why such variation arises among close kin. Here we explore some of the mechanisms and conditions through which task specialization can evolve in groups of related individuals. Recent work on division of labor in insect societies has focused on the self-organization properties of colony behavior. According to a variety of models [8]–[12] colony properties emerge from the behavior of individual workers whose reactions to the environment is governed by simple rules. The behavioral rules leading to emergent specialization are probably shaped by natural selection [10], [13], yet only few studies have focused on the evolution of these rules [14], [15]. Previous work focusing on the benefits of task specialization in other systems (e. g. enzyme-substrate specialization, coordination in co-viruses) generally disregard the mechanisms underlying it, viewing instead specialists and generalists as fixed behavioral strategies [16], [17]. It is thus important to develop models that integrate the evolutionary and self-organization perspective, in order to create a better understanding of division of labor and its evolution [7]. In previous work, we took the response threshold model [1] as a starting point for an evolutionary model for division of labor (A. Duarte, I. Pen, L. Keller and F. J. Weissing, submitted). In the response threshold model, individuals compare an environmental stimulus for a task with their response thresholds; they perform the task if the stimulus is above their threshold, otherwise they remain idle. Using this predefined behavioral architecture, we allowed the evolution of threshold values and showed that division of labor can evolve from a homogeneous population via evolutionary branching, but only if there are clear fitness benefits of individual specialization. Our work also revealed that the response threshold model has the drawback that it imposes severe constraints on the distribution of workers over tasks. Here we look at a more flexible behavioral architecture that is represented by a simple artificial neural network (ANN). ANNs simulate the processing of stimuli by individuals, from stimulus perception by receptor nodes to effector nodes determining the behavioral output [18], [19]. ANNs have been used in evolutionary robotics to understand the evolution of communication and cooperation [20]–[22]. In a recent paper, Lichocki et al. showed that ANN' s, in comparison to response threshold mechanisms, allow for more efficient worker allocation through task switching [23]. Here we examine the effect of the architecture of ANN' s in worker specialization and worker allocation, in a context where task switching is detrimental. In the response threshold model, the response to task-associated stimuli is determined by task-associated thresholds. The stimuli, which reflect the colony' s need for work on the various tasks, change dynamically due to two factors: there is an inherent tendency for the stimuli to increase, and they are decreased whenever the corresponding task is performed. We keep most assumptions of the threshold model but allow the task-associated stimuli to be processed by an ANN. In principle both the architecture of the network and the way information is processed could evolve [24], [25], however, we for simplicity, we focus on predefined architectures (with a fixed number of receptor and effector nodes) and allow only for the evolution of connections between the nodes. The stimuli are processed by an ANN consisting of two receptor nodes and two effector nodes (Figure 1). In a second part of our study, we keep the same network structure but allow for the evolution of a feedback from the effector nodes to the processing of the stimuli (Figure 1C). In other words, an effect of previous experience on current decisions can evolve. An effect of previous experience on task preference, leading to division of labor, has been observed in natural colonies [26], thus it would be interesting to observe under which circumstances it could evolve. We investigate if these slightly more sophisticated mechanisms for processing input allow for the evolution of adaptive division of labor. More precisely, we study whether task specialization among workers can evolve and moreover, whether an appropriate distribution of workers over tasks can be achieved. Throughout, the main question is whether, and to what extent, the evolution of self-organized division of labor is determined by the underlying architecture of behavior.
The first network studied is a simple feedforward network [19] that consists of two stimulus input nodes and two behavioral output nodes, all four nodes being connected (Figure 1B). Each input node perceives a task-associated stimulus with a certain error (drawn from a normal distribution with mean 0 and standard deviation 1). The two signals are then processed and transmitted to the output neurons, via connections with weights that are evolvable properties of the network. Output nodes receive a weighted sum of the stimuli, generally designated activation energy. The activation energy of an output node i is thus: (1) Each output neuron is characterized by a threshold, which is another evolvable property. If the activation energy of an output neuron exceeds the threshold, the neuron is activated, meaning that an individual is willing to perform the respective task. If both output neurons are activated, one task is chosen at random. Note that the response threshold model implemented in previous work is in fact a special case of the feedforward neural network, where and (Figure 1A). The main difference between our feedforward ANN model and the response threshold model is thus the evolution of the connection weights that determine how incoming information is processed and interpreted. The initial values of connection weights in our simulations are: and. Changes in the connection weights and thresholds take place when new individuals are produced, via mutation (see below). During the lifetime of an individual, the parameters of its network are fixed. Thus we do not consider the changing of connection weights with learning, for example. The second network architecture studied is a recurrent network [19]. It includes all previous nodes and connections, and in addition it has two self-feedback loops (Figure 1C). The activation energy in a given time step will affect the activation energy in the next time step: . The connection weight given to the previous activation energy (from here on called the self-feedback connection) is also an evolvable property that changes through mutation and natural selection during production of new individuals. During the lifetime of individuals, however, there is no change occurring in the parameters of the networks. Self-feedback connection weights were initialized at zero, which is equivalent to the feedforward network, without any influence of past experience in current decisions. After the work phase, the fitness of each colony is computed based on how much work the workers performed for each task. Fitness is assumed to be proportional to the weighted geometric mean of work done for both tasks: (2) where is the total number of acts performed for task (A. Duarte, I. Pen, L. Keller, F. J. Weissing, subm.). We take the geometric rather than the arithmetic mean in order to ensure that fitness can only be achieved if both tasks are being performed. The weighing factor allows us to consider the (realistic) situation that not all tasks need to be performed equally often. For the fitness function (2), fitness is maximized if idleness is eliminated (i. e. , if is maximal) and if the workers distribute over tasks according to the ratio. In other words, to maximize fitness the proportion of work allocated to task 1 by the colony should be equal to: (3) Each generation, 2M reproductive offspring are produced in total in the population. Colonies contribute to the population' s pool of sexual individuals in proportion to their fitness. Population size is thus fixed. The reproductive individuals then form M pairs randomly. From each pair one individual will found a new colony with workers, while the old colonies are eliminated. We allowed for the evolution of all connection weights and thresholds of output nodes, giving us in a total 6 (resp. 8) evolving traits. These traits are encoded by 6 (resp. 8) gene loci. The alleles at these loci correspond to real numbers, with threshold alleles being larger or equal to zero, while connection weight alleles may also attain negative values. To keep the genetic assumptions as simple as possible, we assume that all individuals are haploid and that the network of each individual is fully determined by its genotype. Genotypes of workers and sexuals are similarly inherited: Both types of individuals are offspring of the mated colony foundress, and possess alleles for thresholds and connection weights. Our model allows genetic linkage of the threshold loci or linkage of the connection weight loci, but both types of loci are considered to be sufficiently far apart in the genome to make them segregate independently. The degree of linkage is determined by a parameter () that corresponds to a recombination rate. With probability, the threshold alleles (resp. the connection weight alleles) are inherited as a block from one of the two parents; with probability, the parent whose allele is transmitted is chosen independently of what happens at the other loci. Mutation occurs with probability at each locus; when a mutation occurs, the genetic value at that locus is changed by adding a real number to it that is drawn from a normal distribution with mean 0 and standard deviation. In our simulations, we typically used and. We evaluate colony-level characteristics such as the proportion of work devoted to each task and the level of individual specialization. For each individual we calculate at the end of a simulation the fraction of time steps that it stayed in the same task from that time step to the next. We average over all workers and normalize this measure by dividing by the probability that individuals stay in the same task merely due to chance. The latter is given by, where is the proportion of work devoted to task. By subtracting 1 from the value thus obtained, we obtain a measure of worker specialization that ranges between −1 and 1 (A. Duarte, I. Pen, L. Keller and F. J. Weissing, subm.): (4) When is close to 1, there is a high degree of division of labor, and individuals stay in the same task much more often than expected by chance. If is close to zero, workers switch between tasks at random. If is lower than zero, individuals switch task more often than expected by chance. Worker specialization can be adaptive if there is a cost to switching tasks (such as a time cost if tasks are confined to different locations, or a cognitive cost), or if specialized workers perform their task with higher efficiency [27]. Here we implemented a time cost scenario, by imposing time steps of inactivity whenever an individual chooses to switch from one task to the other.
We tested the behavior of a more complex network, where the activation energy of an output neuron could have a feedback on the activation energy at the next time step (Figure 1C). The self-feedback connections were allowed to co-evolve with the rest of the network. We ran ten replicate simulations for all the parameter combinations tested above.
Here we studied whether and how two different neural network architectures enable the evolution of self-organized division of labor and adaptive task ratios. Our results are summarized in table 1. With a feedforward network (table 1), worker specialization evolved more easily (i. e. at lower switching costs) in the absence of recombination. In the absence of recombination the connection weights can co-evolve as a tightly linked block of genes, making it easier to evolve specific combinations of connection weights favoring specialization. Recombination pushes populations into a solution where only one connection weight locus branches, the rest of the network being relatively homogeneous in the population. This allows worker specialization to occur, but to a lesser extent than in the absence of recombination, because at least one of the parent networks in a specialized colony behaves as a generalist for a large range of stimulus combinations. A large percentage of colonies showed no worker specialization, hence, no division of labor. This is because random mating allows for couples with similar genotypes to produce colonies where workers are too similar and therefore division of labor cannot emerge. Previous work on the response threshold model (A. Duarte, I. Pen, L. Keller and F. J. Weissing, subm.) showed that the work ratio could not easily deviate from 1∶1, even if a biased work ratio was optimal. In contrast, in the case of the feedforward network, the work ratio was always biased for one of the tasks, even when a symmetric work ratio was optimal (table 1). Owing to selection for minimizing idleness, the evolved networks maximized the amount of work done by using the stimulus from one of the tasks to stimulate workers to perform the other task. In this way, one of the tasks was performed in excess (the ‘preferred’ task), even when its associated stimulus had been depleted. Although this may seem counter-intuitive, it represents an advantage over networks that attempt to maximize both tasks, because these networks would be limited to the work strictly necessary to reduce stimuli to zero. When, the optimal work ratio was achieved, but only in the absence of switching costs. When switching costs were present, the most common evolved strategy was to increase the proportion of work for task 1 in order to minimize switching among tasks. Some of the limitations of the simple feedforward network were eliminated in the slightly more complex architecture of the recurrent network, where previous activation energies feed back on current activation energies. Worker specialization evolved at low switching costs, now both in the presence and absence of recombination (table 1), at least for. Interestingly, the presence of recombination favored an outcome where all colonies showed a high degree of specialization. In these populations, specialization does not depend on the presence of two complementary networks in the parents of a colony (as in figure 3), but on a strengthening of the self-feedback connections. This allows for initial differences between individuals in stimulus perception to be amplified in subsequent time steps and leads to behavioral differentiation through reinforcement of previous experiences. In the presence of recombination, this strategy prevails. However, when no recombination occurs, evolutionary branching of connection weights is still the prevalent strategy through which worker specialization evolves. Why is the experience-based strategy not observed in all simulations? A likely reason is that to reach this strategy, the values of neural connections must first pass through values where, in the absence of recombination, evolutionary branching is more advantageous. Hence, the evolutionary outcome is dependent on initial conditions. We confirmed this by running simulations where the self-feedback connections were initialized at higher values (e. g. ,); in this case all populations evolved the experience-based strategy rather than evolutionary branching (results not shown). The evolution of an experience-based strategy is affected by stochastic effects at the moment that the population passes the “branching point”, namely on the direction and magnitude of genetic variation, that may lead to local fitness optima. The two strategies may thus represent alternative stable states. The mean population fitness of the genetic specialization (evolutionary branching) is noticeably lower than the mean population fitness of the experience-based strategy (Figure S7). The recurrent network also allowed for the optimal work ratio to be reached in most cases, at least by part of the population (Table 1), even in the presence of switching costs. When, the self-feedback connections allow the continuous activation of both tasks, stimulating individuals that had previously done a task to do it again, even in the absence of the corresponding task stimulus. With this architecture it is also harder to attain the optimal work ratio when and switching costs are considered, and only few replicate populations show both and high degree of worker specialization. The recurrent network has similarities with the reinforced threshold model, in which individual thresholds are lowered after the performance of the respective tasks and increased when the tasks are not performed [9], [29]. In both models, initial differences in experience lead to consistent behavioral differentiation, thus bypassing the need of specific genetic combinations for the emergence of task specialization. However, in terms of the distribution of workers over tasks, the reinforced threshold model suffers from the same limitations as the fixed threshold model, with worker distribution being mainly dependent on the parameters of stimulus dynamics (A. Duarte, T. Janzen, F. J. Weissing and I. Pen, in prep.). Our results highlight the importance of considering asymmetries in models of division of labor. In the evolutionary response threshold model by A. Duarte, I. Pen, L. Keller and F. J. Weissing (subm.), we show that a biased -value cannot be obtained through the evolution of thresholds. To achieve a biased -value in this model, asymmetry must be present in the environment (e. g. in the values of task-associated stimuli [8]) to which the response-threshold mechanism then responds. However, in reality, asymmetries in the work distribution might also arise from the ability of individuals to perceive and prioritize tasks differently. Here we show that, for both types of networks studied, it is not easy to evolve strict worker specialization together with an asymmetric distribution of workers over tasks. A major difficulty is that in case of genetically determined specialization the work proportion is dependent, to a large extent, on the proportions of different specialists in each colony. Since we only consider single-mated foundresses, colonies in our model show either equal proportions of the two specialist strategies or only one of the specialist strategies. Evolving experience-based specialization enables an asymmetric work distribution and division of labor (although at a lower degree of worker specialization than under symmetric conditions, and only in the absence of recombination), yet the trajectory towards this strategy is subject to stochastic effects that may diverge evolution towards genetically determined specialization or towards an increase of performance of the most needed task beyond its optimal level. The observed difficulty in favoring a specific work ratio under switching costs indicates that the simple behavioral architectures investigated are limited in the ability to evolve efficient solutions to complex optimization problems. In the presence of switching costs, it is important for colonies to maximize worker specialization, while at the same time minimizing the number of idle workers and optimizing the work ratio. The behavioral architectures considered thus far were only able to evolve sub-optimal solutions to this multi-faceted problem. Modelling the evolution of behavioral mechanisms by means of artificial neural networks presents several advantages when compared to a priori chosen behavioral architectures such as a response threshold mechanism. First, mechanisms potentially leading to self-organized division of labor are not built into the model, but must emerge from the model. Second, evolving neural networks transcend some limitation of the human mind. When asked to design plausible mechanisms, the imagination of most modellers is limited to simple and intuitive mechanisms (like a response-threshold mechanism) that our mind can easily envisage. For example, it is unlikely that one would envisage a mechanism where a task-associated stimulus does not stimulate the performance of its corresponding task, but of a different one, as it occurs in the feedforward network. By using an independent modelling setup, we can get an idea whether, and to what extent, the results based on the more standard implementations are robust. In our case, the simple feedforward network is too constrained to achieve worker specialization and an appropriate distribution of workers over tasks. By adding a simple elemental feedback the resulting recurrent network had a much higher evolutionary potential. In future models we could consider the evolution of the network' s topology, e. g. by allowing the addition and elimination of neurons and connections to an existing network through mutation [24]. The simple feed-forward neural network was constrained by a problem already present with the response threshold mechanism: to get specialization at the colony level, the coexistence of two specialist genotypes is necessary. Random mating and recombination played an important role in the evolutionary outcome. In general we observed that recombination made it more difficult for genetic specialization to evolve. With recombination, evolutionary branching at multiple loci occurred only rarely, at very high switching costs. This is in accordance with the argument that, in constant environments, recombination may destroy favorable allelic combinations [30], [31]. Our model suggests that in systems where strong genetic task determination and high recombination rates exist, multiple mating would be favored, in order to increase the chance that workers have favorable allelic combinations. This is in accordance to what we observe in honeybees [32], [33]. Under the recurrent network architecture, recombination may also play a beneficial role by creating more genetic variation in the self-feedback connections, which could favor division of labor emerging through the experience-based strategy. The purpose of our approach was not to represent the behavioral architecture of real organisms, but to present a conceptual model that could shed some light on the role of architectural constraints in the evolution of self-organized division of labor. A limitation of this approach is that the larger the network, the more difficult it is to draw conclusions that are biologically relevant. We have implemented two very simple networks, and yet already have six to eight evolvable parameters. We were able to understand the interaction of the networks with the environment and pinpoint the key connections that allowed for specific behaviors, but this may not be possible for more complex architectures. The fitness function used (eq. 2) favored the minimization of idleness. Although it is not unrealistic to assume that more work will translate to higher colony productivity, in reality social insect colonies contain a large proportion of idle workers [34]–[36]. Examples of circumstances that would allow the presence of idle workers include environmental perturbations that require quick recruitment of “stand-by” workers, advantage of energy-saving strategies under poor resource conditions, and selective neutrality of “incompetent” workers due to highly redundant organization of work [36] (and references therein). As stressed before, here we present a conceptual model for the effect of behavioral architectures in division of labor, and necessarily simplify certain assumptions. A more realistic version of our model would treat fitness as the number of offspring produced by a colony, and explicitly consider the nature of the different tasks (e. g. foraging and brood care). Division of labor is a broad topic, with many aspects that were outside the scope of this study. Previous theoretical work has focused on the evolution of differentiated multicellularity, the evolution of germ and soma in multicellular organisms, and the effect of developmental plasticity in gene expression as a cause of individual differentiation [37]–[40]. Here we focused on the evolution of behavioral task specialization in groups where reproductive altruism (analogous to germ-soma differentiation) has already evolved, an assumption which is in line with a recent comparative analysis of the evolutionary history of division of labor [41]. We did not consider the role of developmental plasticity, although this plays an important role in the differentiation of morphological castes [42]. Underlying the different questions concerning division of labor, however, is a problem of functional optimization: Organisms can increase their reproductive success if they perform different tasks efficiently. Dividing tasks among lower-level units within the organism or colony (often referred to as a superorganism) is a solution to the problem. What our model suggests is that the particular behavioral rules through which task specialization arises may impact the evolutionary outcome.
In insect colonies, different individuals specialize in different tasks related to colony maintenance and growth. Unveiling why this division of labor evolved and how individuals decide which task to take on is crucial for our understanding of complex group behavior. Here we model the evolution of general behavioral rules for processing environmental signals of task need in social insect colonies, using artificial neural networks. We examine the patterns of individual specialization that arise in the course of evolution. Division of labor is likely to evolve if switching between tasks decreases worker productivity, but the pattern of division of labor across colonies is highly dependent on the architecture of the networks considered. In networks that allow for a feedback of previous experience on future task choice, division of labor can evolve across the whole population of colonies. In networks where this feedback is not allowed, the presence of division of labor is constrained by the specific genetic composition of colonies. Network architecture also affects how fine-tuned the worker allocation to different tasks can be when the tasks have different requirements.
forms of evolution
theoretical biology
biology
computational biology
evolutionary biology
evolutionary processes
Implications of Behavioral Architecture for the Evolution of Self-Organized Division of Labor