In this work, we present a novel framework for on-line human gait stability prediction of the elderly users of an intelligent robotic rollator using Long Short Term Memory (LSTM) networks, fusing multimodal RGB-D and Laser Range Finder (LRF) data from non-wearable sensors. A Deep Learning (DL) based approach is used for the upper body pose estimation. The detected pose is used for estimating the body Center of Mass (CoM) using Unscented Kalman Filter (UKF). An Augmented Gait State Estimation framework exploits the LRF data to estimate the legs' positions and the respective gait phase. These estimates are the inputs of an encoder-decoder sequence to sequence model which predicts the gait stability state as Safe or Fall Risk walking. It is validated with data from real patients, by exploring different network architectures, hyperparameter settings and by comparing the proposed method with other baselines. The presented LSTM-based human gait stability predictor is shown to provide robust predictions of the human stability state, and thus has the potential to be integrated into a general user-adaptive control architecture as a fall-risk alarm.
The precise analysis of a patient's or an elderly person's walking pattern is very important for an effective intelligent active mobility assistance robot. This walking pattern can be described by a cyclic motion, which can be modeled using the consecutive gait phases. In this paper, we present a completely non-invasive framework for analyzing and recognizing a pathological human walking gait pattern. Our framework utilizes a laser range finder sensor to detect and track the human legs, and an appropriately synthesized Hidden Markov Model (HMM) for state estimation, and recognition of the gait patterns. We demonstrate the applicability of this setup using real data, collected from an ensemble of different elderly persons with a number of pathologies. The results presented in this paper demonstrate that the proposed human data analysis scheme has the potential to provide the necessary methodological (modeling, inference, and learning) framework for a cognitive behavior-based robot control system. More specifically, the proposed framework has the potential to be used for the classification of specific walking pathologies, which is needed for the development of a context-aware robot mobility assistant. A robust and effective gait analysis functionality is an essential characteristic for an assistance mobility robot dealing with elderly persons. The aforementioned functionality is crucial for dealing with mobility disabilities which are widespread in these parts of the population. In this work we present experimental validation of our in house developed system. We are using real data, collected from an ensemble of different elderly persons with a number of pathologies, and we present a validation study by using a GaitRite System. Our system, following the standard literature conventions, characterizes the human motion with a set of parameters which subsequently can be used to assess and distinguish between possible motion disabilities, using a laser range finder as its main sensor. The initial results, presented in this work, demonstrate the applicability of our framework in real test cases. Regarding such frameworks, a crucial technical question is the necessary complexity of the overall tracking system. To answer this question, we compare two approaches with different complexity levels. The first is a static rule based system acting on filtered laser data, while the second system utilizes a Hidden Markov Model for gait cycle estimation, and extraction of the gait parameters. The results demonstrate that the added complexity of the HMM system is necessary for improving the accuracy and efficacy of the system. The accurate human gait tracking is an important factor for various robotic applications, such as robotic walkers aiming to provide assistance to patients with different mobility impairment, social robot companions, etc. A context-aware robot control architecture needs constant knowledge of the user's kinematic state to assess the patient's gait status and adjust its movement properly to provide optimal assistance. In this letter, we present a novel human gait tracking approach that uses two particle filters (PFs) and probabilistic data association (PDA) with an interacting multiple model (IMM) scheme for a real-time selection of the appropriate motion model according to the human gait analysis and the use of the Viterbi algorithm for an augmented human gait state estimation. The gait state estimates also interact with the IMM as a prior information that drives the Markov sampling process, while the PDA ensures that the legs of the same person are coupled. The observation data in this work are provided by a laser range finder mounted on a robotic assistant walker. A detailed experimental validation is presented using ground truth data from a motion capture system, which was used in real experiments with elder subjects who presented various mobility impairments. The validation analysis regards the algorithm's accuracy, robustness to occlusions and clutter, and the gait state classification success, subject to the effect of different number of samples used in the PFs. The results for the elder subjects show the dynamics of the proposed algorithm to be used in a real-time application due to its efficacy to provide accurate and robust augmented human gait estimates with a small number of particles. Abstract For effective gait rehabilitation treatments, the status of a patient’s gait needs to be analyzed precisely. Since the gait motions are cyclic with several gait phases, the gait motions can be analyzed by gait phases. In this paper, a Hidden Markov Model (HMM) is applied to analyze the gait phases in the gait motions. Smart Shoes are utilized to obtain the ground reaction forces (GRFs) as observed data in the HMM. The posterior probabilities from the HMM are used to infer the gait phases, and the abnormal transition between gait phases are checked by the transition matrix. The proposed gait phase analysis methods have been applied to actual gait data, and the results show that the proposed methods have the potential of tools for diagnosing the status of a patient and evaluating a rehabilitation treatment.  In this paper we describe a control strategy for a user-adaptive human-robot system for an intelligent robotic Mobility Assistive Device (MAD)using raw data from a single laser-range-finder (LRF)mounted on the MAD and scanning the walking area. The proposed control architecture consists of three modules. In the first module, a previously proposed methodology (termed IMM-PDA-PF)delivers the augmented human state estimation of the user by providing robust leg tracking and on-line estimation of the human gait phases. This information is processed at the next module for providing the pathological gait parametrization and characterization, by computing specific gait parameters for each gait cycle. These gait parameters form the feature vector that classifies the user in a certain class related to risk of fall. Those are of particular significance to the system, since the gait parameters and the respective class are used in the third module, i.e. the human-robot formation controller, in order to adapt the desired formation of the human-robot system, by selecting the appropriate control variables. The experimental evaluation comprises gait data from real patients, and demonstrates the stability of the human-robot formation control, indicating the importance of incorporating an on-line gait characterization of the user, using non-wearable and non-invasive methods, in the context of a robotic MAD. For an effective intelligent active mobility assistance robot, the walking pattern of a patient or an elderly person has to be analyzed precisely. A well-known fact is that the walking patterns are gaits, that is, cyclic patterns with several consecutive phases. These cyclic motions can be modeled using the consecutive gait phases. In this paper, we present a completely non-invasive framework for analyzing a normal human walking gait pattern. Our framework utilizes a laser range finder sensor to collect the data, a combination of filters to preprocess these data, and an appropriately synthesized Hidden Markov Model (HMM) for state estimation, and recognition of the gait data. We demonstrate the applicability of this setup using real data, collected from an ensemble of different persons. The results presented in this paper demonstrate that the proposed human data analysis scheme has the potential to provide the necessary methodological (modeling, inference, and learning) framework for a cognitive behavior-based robot control system. More specifically, the proposed framework has the potential to be used for the recognition of abnormal gait patterns and the subsequent classification of specific walking pathologies, which is needed for the development of a contextaware robot mobility assistant.