Intelligent Approaches in Locomotion - A Review <s> Hidden Markov <s> “Mimesis” theory focused in the cognitive science field and “mirror neurons” found in the biology field show that the behavior generation process is not independent of the behavior cognition process. The generation and cognition processes have a close relationship with each other. During the behavioral imitation period, a human being does not practice simple joint coordinate transformation, but will acknowledge the parents’ behavior. It understands the behavior after abstraction as symbols, and will generate its self-behavior. Focusing on these facts, we propose a new method which carries out the behavior cognition and behavior generation processes at the same time. We also propose a mathematical model based on hidden Markov models in order to integrate four abilities: (1) symbol emergence; (2) behavior recognition; (3) self-behavior generation; (4) acquiring the motion primitives. Finally, the feasibility of this method is shown through several experiments on a humanoid robot. <s> BIB001 </s> Intelligent Approaches in Locomotion - A Review <s> Hidden Markov <s> This paper describes a novel approach for autonomous and incremental learning of motion pattern primitives by observation of human motion. Human motion patterns are abstracted into a dynamic stochastic model, which can be used for both subsequent motion recognition and generation, analogous to the mirror neuron hypothesis in primates. The model size is adaptable based on the discrimination requirements in the associated region of the current knowledge base. A new algorithm for sequentially training the Markov chains is developed, to reduce the computation cost during model adaptation. As new motion patterns are observed, they are incrementally grouped together using hierarchical agglomerative clustering based on their relative distance in the model space. The clustering algorithm forms a tree structure, with specialized motions at the tree leaves, and generalized motions closer to the root. The generated tree structure will depend on the type of training data provided, so that the most specialized motions will be those for which the most training has been received. Tests with motion capture data for a variety of motion primitives demonstrate the efficacy of the algorithm. <s> BIB002 </s> Intelligent Approaches in Locomotion - A Review <s> Hidden Markov <s> In this paper, a probability-based balance monitoring concept for humanoid robots is proposed. Two algorithms are presented that allow us to distinguish between exceptional situations and normal operations. The first classification approach uses Gaussian-Mixture-Models (GMM) to describe the distribution of the robot's sensor data for typical situations such as stable walking or falling down. With the GMM it is possible to state the probability of the robot being in one of the known situations. The concept of the second algorithm is based on Hidden-Markov-Models (HMM). The objective is to detect and classify unstable situations by means of their typical sequences in the robot's sensor data. When appropriate reflex motions are linked to the critical situations, the robot can prevent most falls or is at least able to execute a controlled falling motion. The proposed algorithms are verified by simulations and experiments with our bipedal robot BARt-UH. <s> BIB003 </s> Intelligent Approaches in Locomotion - A Review <s> Hidden Markov <s> This paper proposes a new mimesis scheme for partial observations, consisting of two strategies; (1) motion understanding from partial observations and (2) proto-symbol-based motion duplication. With the proposed method, whole-body motion imitation is possible even when observing partial motion data. The scheme enables a humanoid robot to imitate a new observed motion by utilizing its own prior knowledge, without learning the demonstrated motion. Evaluation factors, such as inheritance coordinate and matching error, are introduced to evaluate imitation performance. The feasibility of the proposed scheme is demonstrated by an evaluation for a 20-degree-of-freedom humanoid robot. <s> BIB004
Some research has been conducted into robotic learning by imitation, where the robot goal is to observe a motion pattern and then to reproduce it. Often, a human may be the source of the motion pattern to be copied and because the physical workings and capabilities of the source and target systems are different, simple copying will not work. Imitation therefore, becomes a process of observation and re-synthesis. Here, observing is estimating the underlying state variables of the source system where only the output is available, and Hidden Markov Models (HMM) have been used in robotics for this task BIB001 - BIB002 . When imitating, the system first observes a movement pattern and a recognition algorithm is then used to determine if the movement is already known, in which case the pattern is used to refine the stored one. If the pattern is not recognized, then it is added to the database as a new, learned pattern. The observed kinematic or kinetic values, as well as the synthesized values, are called 'motion elements' in the HMM papers reviewed here. One motion element represents the kinematic/kinetic values at one discrete moment of time. The hidden states of the HMM provide an abstraction of the movement patterns, which can be used to re-synthesise the motion in the target system. The first part of imitation is recognition, where a recursive algorithm calculates the probability of observing a movement pattern, if the candidate HMM was used to generate it. If this probability exceeds a threshold value (which can be varied to control grouping of similar observations), then the observation is determined to have fit the stored model and is, therefore, recognized. This recognition algorithm was modified in BIB004 to cope with missing elements. This could be used if observation was temporarily occluded. If recognition does not occur, then the next stage of imitation is learning. A new HMM is generated from the observation sequence using an expectation-maximisation algorithm (EM), such as the Baum-Welch algorithm  . To synthesize, or produce the movement once stored, it is usually generated stochastically from the HMM. Because of its probabilistic nature, typically, the synthesis is repeated several times, with each sequence normalized in time. The sequences are then averaged to produce a final output. In BIB004 , the Viterbi algorithm  was used instead, to generate a sequence that most closely matches the observation. Finally, an error value is generated based on the difference between the synthesized and observed sequences. This error value can then be used in a learning rule to modify the matrices of the HMM and refine the stored pattern. Although typically used for imitation tasks, HMM have also been employed to detect problematic states. For example, in BIB003 , a Gaussian Mixture Model method was used to identify the current state and a HMM method was used to classify a series of states over time. Once a problem, such as falling, was detected, a reflex algorithm was initiated in which both methods were successful with one version of the HMM approach being excellent at fall prevention, or at least producing a controlled fall.