For autonomous robots in dynamic environments mixed with human, it is vital to detect impending collision quickly and robustly. The biological visual systems evolved over millions of years may provide us efficient solutions for collision detection in complex environments. In the cockpit of locusts, two Lobula Giant Movement Detectors, i.e. LGMD1 and LGMD2, have been identified which respond to looming objects rigorously with high firing rates. Compared to LGMD1, LGMD2 matures early in the juvenile locusts with specific selectivity to dark moving objects against bright background in depth while not responding to light objects embedded in dark background - a similar situation which ground vehicles and robots are facing with. However, little work has been done on modeling LGMD2, let alone its potential in robotics and other vision-based applications. In this article, we propose a novel way of modeling LGMD2 neuron, with biased ON and OFF pathways splitting visual streams into parallel channels encoding brightness increments and decrements separately to fulfill its selectivity. Moreover, we apply a biophysical mechanism of spike frequency adaptation to shape the looming selectivity in such a collision-detecting neuron model. The proposed visual neural network has been tested with systematic experiments, challenged against synthetic and real physical stimuli, as well as image streams from the sensor of a miniature robot. The results demonstrated this framework is able to detect looming dark objects embedded in bright backgrounds selectively, which make it ideal for ground mobile platforms. The robotic experiments also showed its robustness in collision detection - it performed well for near range navigation in an arena with many obstacles. Its enhanced collision selectivity to dark approaching objects versus receding and translating ones has also been verified via systematic experiments.
Due to their relatively simple nervous system, insects are an excellent way through which we can investigate how visual information is acquired and processed in order to trigger specific behaviours, as flight stabilization, flying speed adaptation, collision avoidance responses, among others. From the behaviours previously mentioned, we are particularly interested in visually evoked collision avoidance responses. These behaviors are, by necessity, fast and robust, making them excellent systems to study the neural basis of behavior. On the other hand, artificial collision avoidance is a complex task, in which the algorithms used need to be fast to process the captured data and then perform real time decisions. Consequently, neurorobotic models may provide a foundation for the development of more effective and autonomous devices. In this paper, we will focus our attention in the Lobula Giant Movement Detector (LGMD), which is a visual neuron, located in the third layer of the locust optic lobe, that responds selectively to approaching objects, being responsible for trigger collision avoidance maneuvers in locusts. This selectivity of the LGMD neuron to approaching objects seems to result from the dynamics of the network pre-synaptic to this neuron. Tipically, this modelation is done by a conventional Difference of Gaussians (DoG) filter. In this paper, we propose the integration of a different model, an Inversed Difference of Gaussians (IDoG) filter, which preserves the different level of brightness in the captured image, enhancing the contrast at the edges. This change is expected to increase the performance of the LGMD model. Finally, a comparative analysis of both modelations, as well as its effect in the final response of the LGMD neuron, will be performed. The lobula giant movement detector (LGMD) neuron of locusts has been shown to preferentially respond to objects approaching the eye of a locust on a direct collision course. Computer simulations of the neuron have been developed and have demonstrated the ability of mobile robots, interfaced with a simulated LGMD model, to avoid collisions. In this study, a model of the LGMD neuron is presented and the functional parameters of the model identified. Models with different parameters were presented with a range of automotive video sequences, including collisions with cars. The parameters were optimised to respond correctly to the video sequences using a range of genetic algorithms (GAs). The model evolved most rapidly using GAs with high clone rates into a form suitable for detecting collisions with cars and not producing false collision alerts to most non-collision scenes. The lobula giant movement detector (LGMD) is an identified neuron in the locust brain that responds most strongly to the images of an approaching object such as a predator. Its computational model can cope with unpredictable environments without using specific object recognition algorithms. In this paper, an LGMD-based neural network is proposed with a new feature enhancement mechanism to enhance the expanded edges of colliding objects via grouped excitation for collision detection with complex backgrounds. The isolated excitation caused by background detail will be filtered out by the new mechanism. Offline tests demonstrated the advantages of the presented LGMD-based neural network in complex backgrounds. Real time robotics experiments using the LGMD-based neural network as the only sensory system showed that the system worked reliably in a wide range of conditions; in particular, the robot was able to navigate in arenas with structured surrounds and complex backgrounds Abstract The lobula giant movement detector (LGMD) of locusts is a visual interneuron that responds with an increasing spike frequency to an object approaching on a direct collision course. Recent studies involving the use of LGMD models to detect car collisions showed that it could detect collisions, but the neuron produced collision alerts to non-colliding, translating, stimuli in many cases. This study presents a modified model to address these problems. It shows how the neurons pre-synaptic to the LGMD show a remarkable ability to filter images, and only colliding and translating stimuli produce excitation in the neuron. It then integrates the LGMD network with models based on the elementary movement detector (EMD) neurons from the fly visual system, which are used to analyse directional excitation patterns in the biologically filtered images. Combining the information from the LGMD neuron and four directionally sensitive neurons produces a robust collision detection system for a wide range of automotive test situations. In many animal species it is essential to recognize approach predators from complex, dynamic visual scenes and timely initiate escape behavior. Such sophisticated behaviours are often achieved with low neuronal complexity, such as in locusts, suggesting that emulating these biological models in artificial systems would enable the generation of similar complex behaviours with low computational overhead. On the other hand, artificial collision detection is a complex task that requires both real time data acquisition and important features extraction from a captured image. In order to accomplish this task, the algorithms used need to be fast to process the captured data and then perform real time decisions. Taking into account the previous considerations, neurorobotic models may provide a foundation for the development of more effective and autonomous devices robots, based on an improved understanding of the biological basis of adaptive behavior. In this paper, we make a comparative analysis between the new computational model of a locust looming-detecting pathway and the model previously proposed by us. The obtained results proved the improvement provided by the pixel remapping in the model performance. Motion detectors in the locust optic lobe and brain fall into two categories: neu- rones that respond selectively to approaching vs. receding objects and neurones that respond selectively to a particular pattern of image motion over a substantial part of the eye, generated by the locust's own movements through its environment. Neurones from the two categories can be differentiated on the basis of their response to motion at a constant velocity at a fixed distance from the locust: neurones of the first category respond equally well to motion in any direction whereas neurones in the second category respond selectively to one preferred direction of motion. Several of the motion detectors of the first category, responding to approaching objects, share the same input organisation, suggesting that it is important in generating a tuning for approaching objects. Anatomical, physiological, and modelling studies have revealed how the selectivity of the response is generated. The selectivity arises as a result of a critical race between excitation, generated when image edges move out over the eye and delayed inhibition, generated by the same edge movements. For excitation to build up, the velocity and extent of edge motion over the eye must increase rapidly. The ultrastructure of the afferent inputs onto the dendrites of collision sensitive neurones reveals a possible substrate for the interaction between excitation and inhibition. This interpretation is supported by both physiological and immunocytochemical evidence. The input organisation of these neurones has been incorporated into the control structure of a small mobile robot, which success- fully avoids collisions with looming objects. The ecological role of motion detectors of the second category that respond to image motion over a substantial part of the visual field, is discussed as is the input organisation that generates this selective response. The broad tuning of these neurones, particularly at low velocities (0.02° s), suggests they may have a role in navigation during migratory flights at altitude. By contrast, their optimum tuning to high-image velocities suggests these motion detectors are adapted for use in a fast flying insect, which does not spend significant time hovering. Microsc. Res. Tech. 56:256 -269, 2002. © 2002 Wiley-Liss, Inc. Two Lobula Giant Movement Detectors (LGMDs) have been identified in the lobula region of the locust visual system: LGMD1 and LGMD2. LGMD1 had been successfully used in robot navigation to avoid impending collision. LGMD2 also responds to looming stimuli in depth, and shares most the same properties with LGMD1; however, LGMD2 has its specific collision selective responds when dealing with different visual stimulus. Therefore, in this paper, we propose a novel way to model LGMD2, in order to emulate its predicted bio-functions, moreover, to solve some defects of previous LGMD1 computational models. The mechanism of ON and OFF cells, as well as bio-inspired nonlinear functions, are introduced in our model, to achieve LGMD2's collision selectivity. Our model has been tested by a miniature mobile robot in real time. The results suggested this model has an ideal performance in both software and hardware for collision recognition. The lobula giant movement detector (LGMD) is an identified neuron in the locust brain that responds most strongly to the image of an approaching object such as a predator. A computational neural network model based on the structure of the LGMD and its afferent inputs is also able to detect approaching objects. In order for the LGMD network to be used as a robust collision detector for robotic applications, we proposed a new mechanism to enhance the feature of colliding objects before the excitations are gathered by LGMD cell. The new model favours grouped excitation but tends to ignore isolated excitation with selective passing coefficients. Experiments with a Khepera robot showed the proposed collision detector worked in real time in an arena surrounded with blocks. Locusts possess a bilateral pair of uniquely identifiable visual neurons that respond vigorously to the image of an approaching object. These neurons are called the lobula giant movement detectors (LGMDs). The locust LGMDs have been extensively studied and this has lead to the development of an LGMD model for use as an artificial collision detector in robotic applications. To date, robots have been equipped with only a single, central artificial LGMD sensor, and this triggers a non-directional stop or rotation when a potentially colliding object is detected. Clearly, for a robot to behave autonomously, it must react differently to stimuli approaching from different directions. In this study, we implement a bilateral pair of LGMD models in Khepera robots equipped with normal and panoramic cameras. We integrate the responses of these LGMD models using methodologies inspired by research on escape direction control in cockroaches. Using randomised winner-take-all' or steering wheel' algorithms for LGMD model integration, the Khepera robots could escape an approaching threat in real time and with a similar distribution of escape directions as real locusts. We also found that by optimising these algorithms, we could use them to integrate the left and right DCMD responses of real jumping locusts offline and reproduce the actual escape directions that the locusts took in a particular trial. Our results significantly advance the development of an artificial collision detection and evasion system based on the locust LGMD by allowing it reactive control over robot behaviour. The success of this approach may also indicate some important areas to be pursued in future biological research. In principle it appears advantageous for single neurons to perform non-linear operations. Indeed it has been reported that some neurons show signatures of such operations in their electrophysiological response. A particular case in point is the Lobula Giant Movement Detector (LGMD) neuron of the locust, which is reported to locally perform a functional multiplication. Given the wide ramifications of this suggestion with respect to our understanding of neuronal computations, it is essential that this interpretation of the LGMD as a local multiplication unit is thoroughly tested. Here we evaluate an alternative model that tests the hypothesis that the non-linear responses of the LGMD neuron emerge from the interactions of many neurons in the opto-motor processing structure of the locust. We show, by exposing our model to standard LGMD stimulation protocols, that the properties of the LGMD that were seen as a hallmark of local non-linear operations can be explained as emerging from the dynamics of the pre-synaptic network. Moreover, we demonstrate that these properties strongly depend on the details of the synaptic projections from the medulla to the LGMD. From these observations we deduce a number of testable predictions. To assess the real-time properties of our model we applied it to a high-speed robot. These robot results show that our model of the locust opto-motor system is able to reliably stabilize the movement trajectory of the robot and can robustly support collision avoidance. In addition, these behavioural experiments suggest that the emergent non-linear responses of the LGMD neuron enhance the system's collision detection acuity. We show how all reported properties of this neuron are consistently reproduced by this alternative model, and how they emerge from the overall opto-motor processing structure of the locust. Hence, our results propose an alternative view on neuronal computation that emphasizes the network properties as opposed to the local transformations that can be performed by single neurons. There are many ways of building collision-detecting systems. In this paper, we propose a novel collision selective visual neural network inspired by LGMD2 neurons in the juvenile locusts. Such collision-sensitive neuron matures early in the ﬁrst-aged or even hatching locusts, and is only selective to detect looming dark objects against bright background in depth, represents swooping predators, a situation which is similar to ground robots or vehicles. However, little has been done on modeling LGMD2, let alone its potential applications in robotics and other vision-based areas. Compared to other collision detectors, our major contributions are ﬁrst, enhancing the collision selectivity in a bio-inspired way, via constructing a computing efﬁcient visual sensor, and realizing the revealed speciﬁc characteristic sofLGMD2. Second, we applied the neural network to help rearrange path navigation of an autonomous ground miniature robot in an arena. We also examined its neural properties through systematic experiments challenged against image streams from a visual sensor of the micro-robot. The Lobula Giant Movement Detector (LGMD) is a wide-field visual neuron that is located in the Lobula layer of the Locust nervous system. The LGMD increases its firing rate in response to both the velocity of the approaching object and its proximity. It has been found that it can respond to looming stimuli very quickly and can trigger avoidance reactions whenever a rapidly approaching object is detected. It has been successfully applied in visual collision avoidance systems for vehicles and robots. This paper proposes a modified LGMD model that provides additional movement depth direction information. The proposed model retains the simplicity of the previous neural network model, adding only a few new cells. It has been tested on both simulated and recorded video data sets. The experimental results shows that the modified model can very efficiently provide stable information on the depth direction of movement. In this paper, we proposed a method for near range path navigation for a mobile robot by using a pair of biologically inspired visual neural network - lobula giant movement detector (LGMD). In the proposed binocular style visual system, each LGMD processes images covering a part of the wide field of view and extracts relevant visual cues as its output. The outputs from the two LGMDs are compared and translated into executable motor commands to control the wheels of the robot in real time. Stronger signal from the LGMD in one side pushes the robot away from this side step by step; therefore, the robot can navigate in a visual environment naturally with the proposed vision system. Our experiments showed that this bio-inspired system worked well in different scenarios. Detecting colliding objects in complex dynamic scenes is a difficult task for conventional computer vision techniques. However, visual processing mechanisms in animals such as insects may provide very simple and effective solutions for detecting colliding objects in complex dynamic scenes. In this paper, we propose a robust collision detecting system, which consists of a lobula giant movement detector (LGMD) based neural network and a translating sensitive neural network (TSNN), to recognise objects on a direct collision course in complex dynamic scenes. The LGMD based neural network is specialized for recognizing looming objects that are on a direct collision course. The TSNN, which fuses the extracted visual motion cues from several whole field direction selective neural networks, is only sensitive to translating movements in the dynamic scenes. The looming cue and translating cue revealed by the two specialized visual motion detectors are fused in the present system via a decision making mechanism. In the system, the LGMD plays a key role in detecting imminent collision; the decision from TSNN becomes useful only when a collision alarm has been issued by the LGMD network. Using driving scenarios as an example, we showed that the bio-inspired system can reliably detect imminent colliding objects in complex driving scenes. In this paper, we proposed a visually stimulated motor control (VSMC) system for autonomous navigation of mobile robots. Inspired from a locusts’ motion sensitive interneuron – lobula giant movement detector (LGMD), the presented VSMC system enables a robot exploring local paths or interacting with dynamic objects effectively using visual input only. The VSMC consists of a pair of LGMD visual neural networks and a simple motor command generator. Each LGMD processes images covering part of the wide field of view and extracts relevant visual cues. The outputs from the two LGMDs are compared and interpreted into executable motor commands directly. These motor commands are then executed by the robot’s wheel control system in real-time to generate corresponded motion adjustment accordingly. Our experiments showed that this bio-inspired VSMC system worked well in different scenarios.