Intelligent Approaches in Locomotion - A Review <s> Conclusion <s> A set of van der Pol oscillators is arranged in a network in which each oscillator is coupled to each other oscillator. Through the selection of coupling coefficients, the network is made to appear as a ring and as a chain of coupled oscillators. Each oscillator is provided with amplitude, frequency, and offset parameters which have analytically indeterminable effects on the output waves. These systems are simulated on the digital computer in order to study the amplitude, frequency, offset, and phase relationships of the waves versus parameter changes. Based on the simulations, systems of coupled oscillators are configured so that they exhibit stable patterns of signals which can be used to model the central pattern generator (CPG) of living organisms. Using a simple biped as an example locomotory system, the CPG model generates control signals for simulated walking and jumping maneuvers. It is shown that with parameter adjustments, as guided by the simulations, the model can be made to generate kinematic trajectories which closely resemble those for the human walking gait. Further-more, minor tuning of these parameters along with some algebraic sign changes of coupling coefficients can effect a transition in the trajectories to those of a two-legged hopping gait. The generalized CPG model is shown to be versatile enough that it can also generate various n-legged gaits and spinal undulatory motions, as in the swimming motions of a fish. <s> BIB001 </s> Intelligent Approaches in Locomotion - A Review <s> Conclusion <s> The authors have developed a movement pattern generator, using an artificial neural network (ANN) for generating periodic movement trajectories. This model is based on the concept of 'central pattern generators'. Jordan's (1986) sequential network, which is capable of learning sequences of patterns, was modified and used to generate several bipedal trajectories (or gaits), coded in task space, at different frequencies. The network model successfully learned all of the trajectories presented to it. The model has many attractive properties, such as limit cycle behavior, generalization of trajectories and frequencies, phase maintenance, and fault tolerance. The movement pattern generator model is potentially applicable for improved understanding of animal locomotion and for use in legged robots and rehabilitation medicine. > <s> BIB002 </s> Intelligent Approaches in Locomotion - A Review <s> Conclusion <s> We would like the behavior of the artificial agents that we construct to be as well-adapted to their environments as natural animals are to theirs. Unfortunately, designing controllers with these properties is a very difficult task. In this article, we demonstrate that continuous-time recurrent neural networks are a viable mechanism for adaptive agent control and that the genetic algorithm can be used to evolve effective neural controllers. A significant advantage of this approach is that one need specify only a measure of an agent's overall performance rather than the precise motor output trajectories by which it is achieved. By manipulating the performance evaluation, one can place selective pressure on the development of controllers with desired properties. Several novel controllers have been evolved, including a chemotaxis controller that switches between different strategies depending on environmental conditions, and a locomotion controller that takes advantage of sensory feedback if available but th... <s> BIB003 </s> Intelligent Approaches in Locomotion - A Review <s> Conclusion <s> The learning of complex control behaviour of autonomous mobile robots is one of the actual research topics. In this article an intelligent control architecture is presented which integrates learning methods and available domain knowledge. This control architecture is based on Reinforcement Learning and allows continuous input and output parameters, hierarchical learning, multiple goals, self-organized topology of the used networks and online learning. As a testbed this architecture is applied to the six-legged walking machine LAURON to learn leg control and leg coordination. <s> BIB004 </s> Intelligent Approaches in Locomotion - A Review <s> Conclusion <s> This paper describes a new method for generating optimal path and gait simultaneously of a six-legged robot using a combined GA-fuzzy approach. The problem of combined path and gait generations involves three steps, namely determination of vehicle’s trajectory, foothold selection and design of a sequence of leg movements. It is a complicated task and no single traditional approach is found to be successful in handling this problem. Moreover, the traditional approaches do not consider optimization issues, yet they are computationally expensive. Thus, the generated path and gaits may not be optimal in any sense. To solve such problems optimally, there is still a need for the development of an efficient and computationally faster algorithm. In the proposed genetic-fuzzy approach, optimal path and gaits are generated by using fuzzy logic controllers (FLCs) and genetic algorithms (GAs) are used to find optimized FLCs. The optimization is done off-line on a number of training scenarios and optimal FLCs are found. The hexapod can then use these GA-tuned FLCs to navigate in test-case scenarios. <s> BIB005 </s> Intelligent Approaches in Locomotion - A Review <s> Conclusion <s> Abstract Gait generation for legged vehicles has since long been considered as an area of keen interest by the researchers. Soft computing is an emerging technique, whose utility is more stressed, when the problems are ill-defined, difficult to model and exhibit large scale solution spaces. Gait generation for legged vehicles is a complex task. Therefore, soft computing can be applied to solve it. In this work, gait generation problem of a two-legged robot is modeled using a fuzzy logic controller (FLC), whose rule base is optimized offline, using a genetic algorithm (GA). Two different GA-based approaches (to improve the performance of FLC) are developed and their performances are compared to that of manually constructed FLC. Once optimized, the FLCs will be able to generate dynamically stable gait of the biped. As the CPU-time of the algorithm is found to be only 0.002 s in a P-III PC, the algorithm is suitable for on-line (real-time) implementations. <s> BIB006 </s> Intelligent Approaches in Locomotion - A Review <s> Conclusion <s> A challenging task that must be accomplished for every legged robot is creating the walking and running behaviors needed for it to move. In this paper we describe our system for autonomously evolving dynamic gaits on two of Sony's quadruped robots. Our evolutionary algorithm runs on board the robot and uses the robot's sensors to compute the quality of a gait without assistance from the experimenter. First, we show the evolution of a pace and trot gait on the OPEN-R prototype robot. With the fastest gait, the robot moves at over 10 m/min, which is more than forty body-lengths/min. While these first gaits are somewhat sensitive to the robot and environment in which they are evolved, we then show the evolution of robust dynamic gaits, one of which is used on the ERS-110, the first consumer version of AIBO. <s> BIB007 </s> Intelligent Approaches in Locomotion - A Review <s> Conclusion <s> Inspired by the efficient method of locomotion of the rattlesnake Crotalus cerastes, the objective of this work is automatic design through genetic programming (GP) of the fastest possible (sidewinding) locomotion of simulated limbless, wheelless snake-like robot (Snakebot). The realism of simulation is ensured by employing the Open Dynamics Engine (ODE), which facilitates implementation of all physical forces, resulting from the actuators, joints constrains, frictions, gravity, and collisions. Reduction of the search space of the GP is achieved by representation of Snakebot as a system comprising identical morphological segments and by automatic definition of code fragments, shared among (and expressing the correlation between) the evolved dynamics of the vertical and horizontal turning angles of the actuators of Snakebot. Empirically obtained results demonstrate the emergence of sidewinding locomotion from relatively simple motion patterns of morphological segments. Robustness of the sidewinding Snakebot, which is considered to be the ability to retain its velocity when situated in an unanticipated environment, is illustrated by the ease with which Snakebot overcomes various types of obstacles such as a pile of or burial under boxes, rugged terrain, and small walls. The ability of Snakebot to adapt to partial damage by gradually improving its velocity characteristics is discussed. Discovering compensatory locomotion traits, Snakebot recovers completely from single damage and recovers a major extent of its original velocity when more significant damage is inflicted. Exploring the opportunity for automatic design and adaptation of a simulated artifact, this work could be considered as a step toward building real Snakebots, which are able to perform robustly in difficult environments. <s> BIB008 </s> Intelligent Approaches in Locomotion - A Review <s> Conclusion <s> This paper proposes a universal stability criterion of the foot contact of legged robots. The proposed method checks if the sum of the gravity and the inertia wrench applied to the COG of the robot, which is proposed to be the stability criterion, is inside the polyhedral convex cone of the contact wrench between the feet of a robot and its environment. The criterion can be used to determine the strong stability of the foot contact when a robot walks on an arbitrary terrain and/or when the hands of the robot are in contact with it under the sufficient friction assumption. The determination is equivalent to check if the ZMP is inside the support polygon of the feet when the robot walks on a horizontal plane with sufficient friction. The criterion can also be used to determine if the foot contact is sufficiently weakly stable when the friction follows a physical law. Therefore, the proposed criterion can be used to judge what the ZMP can, and it can be used in more universal cases <s> BIB009 </s> Intelligent Approaches in Locomotion - A Review <s> Conclusion <s> Abstract This paper reports on experiments involving a hexapod robot. Motivated by neurobiological evidence that control in real hexapod insects is distributed leg-wise, we investigated two approaches to learning distributed controllers: genetic algorithms and reinforcement learning. In the case of reinforcement learning, a new learning algorithm was developed to encourage cooperation between legs. Results from both approaches are presented and compared. <s> BIB010 </s> Intelligent Approaches in Locomotion - A Review <s> Conclusion <s> Research on humanoid robotics in Mechatronics and Automation (MA) Laboratory, Electrical and Computer Engineering (ECE), National University of Singapore (NUS) was started at the beginning of this decade. Various research prototypes for humanoid robots have been designed and are going through evolution over these years. These humanoids have been successfully participating in various robotic soccer competitions. In this paper, three major research and development aspects of the above humanoid research are discussed. The paper focuses on various practical and theoretical considerations involved in processing architecture, gait generation and vision systems. <s> BIB011 </s> Intelligent Approaches in Locomotion - A Review <s> Conclusion <s> The ability to learn during continuous and on-line observation would be advantageous for humanoid robots, as it would enable them to learn during co-location and interaction in the human environment. However, when motions are being learned and clustered on-line, there is a trade-off between classification accuracy and the number of training examples, resulting in potential misclassifications both at the motion and hierarchy formation level. This article presents an approach enabling fast on-line incremental learning, combined with an incremental memory consolidation process correcting initial misclassifications and errors in organization, to improve the stability and accuracy of the learned motions, analogous to the memory consolidation process following motor learning observed in humans. Following initial organization, motions are randomly selected for reclassification, at both low and high levels of the hierarchy. If a better reclassification is found, the knowledge structure is reorganized to comply. The approach is validated during incremental acquisition of a motion database containing a variety of full body motions.1 <s> BIB012 </s> Intelligent Approaches in Locomotion - A Review <s> Conclusion <s> To generate coordinated periodic movements, robot locomotion demands mechanisms which are able to learn and produce stable rhythmic motion in a controllable way. Because systems based on biological central pattern generators (CPGs) can cope with these demands, these kind of systems are gaining in success. In this work we introduce a novel methodology that uses the dynamics of a randomly connected recurrent neural network for the design of CPGs. When a randomly connected recurrent neural network is excited with one or more useful signals, an output can be trained by learning an instantaneous linear mapping of the neuron states. This technique is known as reservoir computing (RC). We will show that RC has the necessary capabilities to be fruitful in designing a CPG that is able to learn human motion which is applicable for imitation learning in humanoid robots. <s> BIB013 </s> Intelligent Approaches in Locomotion - A Review <s> Conclusion <s> In this paper, a probability-based balance monitoring concept for humanoid robots is proposed. Two algorithms are presented that allow us to distinguish between exceptional situations and normal operations. The first classification approach uses Gaussian-Mixture-Models (GMM) to describe the distribution of the robot's sensor data for typical situations such as stable walking or falling down. With the GMM it is possible to state the probability of the robot being in one of the known situations. The concept of the second algorithm is based on Hidden-Markov-Models (HMM). The objective is to detect and classify unstable situations by means of their typical sequences in the robot's sensor data. When appropriate reflex motions are linked to the critical situations, the robot can prevent most falls or is at least able to execute a controlled falling motion. The proposed algorithms are verified by simulations and experiments with our bipedal robot BARt-UH. <s> BIB014 </s> Intelligent Approaches in Locomotion - A Review <s> Conclusion <s> This paper deals with the generation of dynamically balanced gaits of a ditch-crossing biped robot having seven degrees of freedom (DOFs). Three different approaches, namely analytical, neural network (NN)-based and fuzzy logic (FL)-based, have been developed to solve the said problem. The former deals with the analytical modeling of the ditch-crossing gait of a biped robot, whereas the latter two approaches aim to maximize the dynamic balance margin of the robot and minimize the power consumption during locomotion, after satisfying a constraint stating that the changes of joint torques should lie within a pre-specified value to ensure its smooth walking. It is to be noted that the power consumption and dynamic balance of the robot are also dependent on the position of the masses on various links and the trajectory followed by the hip joint. A genetic algorithm (GA) is used to provide training off-line, to the NN-based and FL-based gait planners developed. Once optimized, the planners will be able to generate the optimal gaits on-line. Both the NN-based and FL-based gait planners are able to generate more balanced gaits and that, too, at the cost of lower power consumption compared to those yielded by the analytical approach. The NN-based and FL-based approaches are found to be more adaptive compared to the other approach in generating the gaits of the biped robot. <s> BIB015
The suggestions in this section, although following informed arguments, are hampered by a lack of research publications directly comparing different techniques. This section could be used as a starting point but obviously there are always other methods to try. Despite being relatively scarce, some direct comparisons have been made between different approaches:  A simple oscillator CPG was found to produce faster movement than a ZMP inverted pendulum model method in BIB011 ;  A NN approach appeared to be more stable and efficient than a fuzzy logic approach for a ditch crossing robot in BIB015 . The fuzzy logic method was in turn found to be better than a ZMP IPM technique. A continuum of approaches exists to produce the correct movement, from those that rely fully on prior knowledge and require no training, to those that are very generic pattern generators that require intensive training (Figure 1) . A summary of the highlighted advantages and disadvantages is given in Table 3 . Researchers' preference must also influence the decision. Analytical techniques can become very complex in more demanding situations. However, extensions such as the CWS BIB009 show that the analytical approach is capable of dealing with difficult environments. Conversely, RNNs may be more difficult to develop than the analytical solutions for simple situations, but their generality means a solution should still be available. If a researcher is knowledgeable in one domain but not the other it may be worthwhile searching for solutions using those methods even if another approach is theoretically more suitable. The first criterion for selecting an approach is the availability of prior knowledge. If none is available, then an analytical approach is usually ruled out because of the need for template foot trajectories (analysis is used to provide stability constraints). If prior knowledge is available, and the researcher wishes to use it, then wholly generic approaches such as RNNs, generic CPGs and Markov chains could be ruled out. If the problem is well understood, then a primarily analytical approach may be appropriate. If only a limited amount of prior knowledge is available, then this can still be used in customising the structure of the control or setting some parameter values. In this review, CPGs and NNs have been found to offer the option to evolve or learn solutions via reinforcement or supervision. This frees the designer from having to specify too much, as would be the case for an analytical approach. NNs offer a more generic option (compared to the oscillator primitives of CPGs), with RNNs being the most generic (allowing a combination of internally generated and input driven patterns). However, this generality comes at the expense of more difficult training. Oscillator based CPGs provide a more constrained option that can help reduce the training load. Transition tables should be noted for the ease of training, but they are limited in the kind of information they can generate. To a certain extent all approaches could allow for the development of multiple skills or gaits through the use of a database of parameter values (as is common in HMM approaches BIB012 ). However, analytical approaches will need a new analysis for each skill. CPGs and RNNs have shown examples of emergent gaits and smooth skill transitions as a natural consequence of their limit cycle behaviour  , BIB001 , BIB002 , BIB013 . Hidden Markov Model systems are used for observation and classification as well as learning and generation. They are therefore an option for applications of learning through imitation BIB012 , and initiating reflexes BIB014 . Rule based systems such as transition tables or fuzzy logic controllers have the advantage of clearly interpretable processes, even if they are developed through an optimisation algorithm. The transition table method is suitable for simple situations, with the fuzzy logic controller being more fault tolerant and appropriate for more complex situations BIB015 . Reinforcement learning allows effective control to be learned in situations where uncertainty exists, where a lack of input-output data prevents the use of supervised learning. However, reinforcement learning can often be slow, which often forces learning to be through computer simulation rather than in the real world. Systems that allow a hybrid of expert knowledge plus reinforcement learning give the option of speeding up the process by giving the system a head start. Options include RBFNs and fuzzy logic controllers, where networks or rule based systems are sufficiently interpretable in a sense that an expert can specify an initial solution, as investigated in BIB004 , BIB005 . Although the less constrained methods potentially offer a greater range of solutions, their parameterisation can become complicated. As an example, Table 4 shows the large number of simulations often required for genetic algorithm based solutions (a popular choice). The table does not show the amount of time per simulation and this can vary largely between tasks. However, it can readily be seen that a great amount of simulation time is often required when control methods are general and need extensive parameterisation. Table 4 : Number of simulations used in evolving solutions using genetic algorithms, in a sample of publications covering rule based, CPG and neural network control methods. Reference Method Simulations BIB015 Rule based 900 BIB010 Rule based 1150 BIB006 Rule based 7200 BIB005 Rule based 5000 BIB007 CPG 500 BIB008 CPG 5000 BIB003 Neural network 40000