In this work, we present a novel framework for on-line human gait stability prediction of the elderly users of an intelligent robotic rollator using Long Short Term Memory (LSTM) networks, fusing multimodal RGB-D and Laser Range Finder (LRF) data from non-wearable sensors. A Deep Learning (DL) based approach is used for the upper body pose estimation. The detected pose is used for estimating the body Center of Mass (CoM) using Unscented Kalman Filter (UKF). An Augmented Gait State Estimation framework exploits the LRF data to estimate the legs' positions and the respective gait phase. These estimates are the inputs of an encoder-decoder sequence to sequence model which predicts the gait stability state as Safe or Fall Risk walking. It is validated with data from real patients, by exploring different network architectures, hyperparameter settings and by comparing the proposed method with other baselines. The presented LSTM-based human gait stability predictor is shown to provide robust predictions of the human stability state, and thus has the potential to be integrated into a general user-adaptive control architecture as a fall-risk alarm.
Regarding the extraction of gait motions, different types of sensors have been used @cite_27 @cite_49 . Gait analysis can be achieved by using Hidden Markov Models for modelling normal @cite_17 and pathological human gait @cite_26 , and extracting gait parameters @cite_28 . Recently, we have developed a new method for online augmented human state estimation, that uses Interacting Multiple Model Particle Filters with Probabilistic Data Association @cite_24 , which tracks the users' legs using data from a LRF, while it provides real-time gait phase estimation. We have also presented a new human-robot formation controller that utilizes the gait status characterization for user adaptation towards a fall preventing system @cite_13 .
@cite_26 @cite_28 @cite_24 @cite_27 @cite_49 @cite_13 @cite_17
2794378256 2131737076 2041278045 2197966930
The accurate human gait tracking is an important factor for various robotic applications, such as robotic walkers aiming to provide assistance to patients with different mobility impairment, social robot companions, etc. A context-aware robot control architecture needs constant knowledge of the user's kinematic state to assess the patient's gait status and adjust its movement properly to provide optimal assistance. In this letter, we present a novel human gait tracking approach that uses two particle filters (PFs) and probabilistic data association (PDA) with an interacting multiple model (IMM) scheme for a real-time selection of the appropriate motion model according to the human gait analysis and the use of the Viterbi algorithm for an augmented human gait state estimation. The gait state estimates also interact with the IMM as a prior information that drives the Markov sampling process, while the PDA ensures that the legs of the same person are coupled. The observation data in this work are provided by a laser range finder mounted on a robotic assistant walker. A detailed experimental validation is presented using ground truth data from a motion capture system, which was used in real experiments with elder subjects who presented various mobility impairments. The validation analysis regards the algorithm's accuracy, robustness to occlusions and clutter, and the gait state classification success, subject to the effect of different number of samples used in the PFs. The results for the elder subjects show the dynamics of the proposed algorithm to be used in a real-time application due to its efficacy to provide accurate and robust augmented human gait estimates with a small number of particles. Human motion analysis has received a great attention from researchers in the last decade due to its potential use in different applications. We propose a new approach to extract human joints (vertex positions) using a model-based method. Motion templates describing the motion of the joints as derived by gait analysis, are parametrized using the elliptic Fourier descriptors. The heel strike data is exploited to reduce the dimensionality of the parametric models. People walk normal to the viewing plane, as major gait information is available in a sagittal view. The ankle, knee and hip joints are successfully extracted with high accuracy for indoor and outdoor data. In this way, we have established a baseline analysis which can be deployed in recognition, marker-less analysis and other areas. The experimental results confirmed the robustness of the proposed method to recognize walking subjects with a correct classification rate of 92. For an effective intelligent active mobility assistance robot, the walking pattern of a patient or an elderly person has to be analyzed precisely. A well-known fact is that the walking patterns are gaits, that is, cyclic patterns with several consecutive phases. These cyclic motions can be modeled using the consecutive gait phases. In this paper, we present a completely non-invasive framework for analyzing a normal human walking gait pattern. Our framework utilizes a laser range finder sensor to collect the data, a combination of filters to preprocess these data, and an appropriately synthesized Hidden Markov Model (HMM) for state estimation, and recognition of the gait data. We demonstrate the applicability of this setup using real data, collected from an ensemble of different persons. The results presented in this paper demonstrate that the proposed human data analysis scheme has the potential to provide the necessary methodological (modeling, inference, and learning) framework for a cognitive behavior-based robot control system. More specifically, the proposed framework has the potential to be used for the recognition of abnormal gait patterns and the subsequent classification of specific walking pathologies, which is needed for the development of a contextaware robot mobility assistant. The precise analysis of a patient's or an elderly person's walking pattern is very important for an effective intelligent active mobility assistance robot. This walking pattern can be described by a cyclic motion, which can be modeled using the consecutive gait phases. In this paper, we present a completely non-invasive framework for analyzing and recognizing a pathological human walking gait pattern. Our framework utilizes a laser range finder sensor to detect and track the human legs, and an appropriately synthesized Hidden Markov Model (HMM) for state estimation, and recognition of the gait patterns. We demonstrate the applicability of this setup using real data, collected from an ensemble of different elderly persons with a number of pathologies. The results presented in this paper demonstrate that the proposed human data analysis scheme has the potential to provide the necessary methodological (modeling, inference, and learning) framework for a cognitive behavior-based robot control system. More specifically, the proposed framework has the potential to be used for the classification of specific walking pathologies, which is needed for the development of a context-aware robot mobility assistant.