This paper presents a technique called evolving self-supervised neural networks - neural networks that can teach themselves, intrinsically motivated, without external supervision or reward. The proposed method presents some sort-of paradigm shift, and differs greatly from both traditional gradient-based learning and evolutionary algorithms in that it combines the metaphor of evolution and learning, more specifically self-learning, together, rather than treating these phenomena alternatively. I simulate a multi-agent system in which neural networks are used to control autonomous foraging agents with little domain knowledge. Experimental results show that only evolved self-supervised agents can demonstrate some sort of intelligent behaviour, but not evolution or self-learning alone. Indications for future work on evolving intelligence are also presented.
Nolfi and his colleagues made a simulation of , or robots, controlled by neural networks situated in a grid-world, with discrete state and action spaces @cite_32 . Each agent lives in its own copy of the world, hence no mutual interaction. The evolutionary task is to evolve action strategies to collect food effectively, while each agent learns to predict the sensory inputs to neural networks for each time step. Learning was implemented using backpropagation based on the error between the actual and the predicted sensory inputs to update the weights of a neural network. It was shown that learning to predict can enhance the evolutionary search, hence increasing the performance of the robot.
@cite_32
1979356863 2043369301 2038794597 2079690958
This article describes simulations on populations of neural networks that both evolve at the population level and learn at the individual level. Unlike other simulations, the evolutionary task (finding food in the environment) and the learning task (predicting the next position of food on the basis of present position and planned network's movement) are different tasks. In these conditions, learning influences evolution (without Lamarckian inheritance of learned weight changes) and evolution influences learning. Average but not peak fitness has a better evolutionary growth with learning than without learning. After the initial generations, individuals that learn to predict during life also improve their food-finding ability during life. Furthermore, individuals that inherit an innate capacity to find food also inherit an innate predisposition to learn to predict the sensory consequences of their movements. They do not predict better at birth, but they do learn to predict better than individuals of the ini... This paper explores the possibility of providing robots with an 'inner world' based on internal simulation of perception rather than an explicit representational world model. First a series of initial experiments is discussed, in which recurrent neural networks were evolved to control collision-free corridor following behavior in a simulated Khepera robot and predict the next time step's sensory input as accurately as possible. Attempts to let the robot act blindly, i.e. repeatedly using its own prediction instead of the real sensory input, were not particularly successful. This motivated the second series of experiments, on which this paper focuses. A feed-forward network was used which, as above, controlled behavior and predicted sensory input. However, weight evolution was now guided by the sole fitness criterion of successful, 'blindfolded' corridor following behavior, including timely turns, as above using as input only own sensory predictions rather than actual sensory input. The trained robot is in some cases actually able to move blindly in a simple environment for hundreds of time steps, successfully handling several multi-step turns. Somewhat surprisingly, however, it does so based on self-generated input that is not particularly similar to the actual sensory values. The idea of using evolutionary computation to train artificial neural networks, or neuroevolution (NE), for reinforcement learning (RL) tasks has now been around for over 20 years. However, as RL tasks become more challenging, the networks required become larger, as do their genomes. But, scaling NE to large nets (i.e. tens of thousands of weights) is infeasible using direct encodings that map genes one-to-one to network components. In this paper, we scale-up our compressed network encoding where network weight matrices are represented indirectly as a set of Fourier-type coefficients, to tasks that require very-large networks due to the high-dimensionality of their input space. The approach is demonstrated successfully on two reinforcement learning tasks in which the control networks receive visual input: (1) a vision-based version of the octopus control task requiring networks with over 3 thousand weights, and (2) a version of the TORCS driving game where networks with over 1 million weights are evolved to drive a car around a track using video images from the driver's perspective. Evolutionary Robotics (ER) is one of promising approaches to design robot controllers which essentially have complicated and or complex properties. In most ER research, the sensory-motor mappings of robots are represented as artificial neural networks, and their connection weights (and sometimes the structure of the networks) can be optimized in the parameter spaces by using evolutionary computation. However, generally, the evolved neural controllers could be fragile in unexperienced environments, especially in real worlds, because the evolutionary optimization processes would be executed in idealized simulators. This is known as the gap problem between the simulated and real worlds. To overcome this, the author focused on evolving an on-line learning ability instead of weight parameters in a simulated environment. According to recent biological findings, actually, the kinds of on-line adaptation abilities can be found in real nervous systems of insects and crustaceans, and it is also known that a variety of neuromodulators (NMs) play crucial roles to regulate the network characteristics (i.e. activating blocking changing of synaptic connections). Based on this, a neuromodulatory neural network model was proposed and it was utilized as a mobile robot controller. In the paper, the detail behavior analysis of the evolved neuromodulatory neural network is also discussed.