Intelligent Approaches in Locomotion - A Review <s> CMAC <s> This paper presents some results from a study of biped dynamic walking using reinforcement learning. During this study a hardware biped robot was built, a new reinforcement learning algorithm as well as a new learning architecture were developed. The biped learned dynamic walking without any previous knowledge about its dynamic model. The self scaling reinforcement (SSR) learning algorithm was developed in order to deal with the problem of reinforcement learning in continuous action domains. The learning architecture was developed in order to solve complex control problems. It uses different modules that consist of simple controllers and small neural networks. The architecture allows for easy incorporation of new modules that represent new knowledge, or new requirements for the desired task. <s> BIB001 </s> Intelligent Approaches in Locomotion - A Review <s> CMAC <s> Abstract In this paper, we propose a control strategy allowing us to perform the dynamic walking gait of an under-actuated robot even if this one is subjected to destabilizing external disturbances. This control strategy is based on two stages. The first one consists of using a set of pragmatic rules in order to generate a succession of passive and active phases allowing us to perform a dynamic walking gait of the robot. The joint trajectories of this reference gait are learned by using neural networks. In the second stage, we use these neural networks to generate the learned trajectories during the first stage. The goal of the use of these neural networks is to increase the robustness of the control of the dynamic walking gait of this robot in the case of external disturbances. The first experimental results are also presented. <s> BIB002 </s> Intelligent Approaches in Locomotion - A Review <s> CMAC <s> This paper presents the main simulation and experimental results from studies of the robustness of a proposed new control strategy for the under-actuated robot RABBIT. The disturbances studied include modifications of the characteristics of the foot/ground interaction and the application of external forces to the trunk of the robot. These two kinds of disturbance correspond to those most frequently encountered by biped robots during walking in an unknown environment. <s> BIB003
A Cerebellar Model Articulation Controller or CMAC network is a type of associative memory network based on the cerebellum  . The continuous input space is divided into hyper-rectangles so that an input is located in one rectangle at any one time. Multiple layers are used with the placement of the rectangles slightly offset for each layer, so a rectangle in one layer will overlap several in the other layers. In this way one input is associated with multiple hyper-rectangles, one in each layer, but changes in the input will result in different changes in activation in each layer. Each hyper-rectangle in each layer has a weighted connection to the output neurons. The output of each node is the weighted sum of the activated rectangles, and the weights are adjusted through training. In BIB001 , a set of CMACs were used to generate locomotion. At the core was a pattern generator formed by a CMAC that took time, step length and walking period as inputs, and generated six desired joint angles as outputs, and was trained using least mean squares supervized learning to match a basic joint trajectory profile over one walk period. The system also used CMACs as peripheral controllers that helped maintain body posture and body height. Additional reinforcement learning was used by the pattern generator and peripheral controllers, based on several constraints: energy, step, posture and a body height constraint. The last two constraints were themselves calculated by CMACs that were trained to assess the current posture and body height states, and to output an appropriate evaluation. The additional learning enabled the robot to refine its movement and learn new solutions. The modular nature of the solution allowed for smaller, easier to train neural networks. In BIB003 , BIB002 CMACs were used to learn walking patterns derived by an analytical approach. The CMACs successfully learned the movement patterns and showed resilience to perturbation (including on uneven or slippery floors), thereby translating a rigid analytical solution into an adaptable one.