The article proposes a new framework for assessment of physical rehabilitation exercises based on a deep learning approach. The objective of the framework is automated quantification of patient performance in completing prescribed rehabilitation exercises, based on captured whole-body joint trajectories. The main components of the framework are metrics for measuring movement performance, scoring functions for mapping the performance metrics into numerical scores of movement quality, and deep neural network models for regressing quality scores of input movements via supervised learning. Furthermore, an overview of the existing methods for modeling and evaluation of rehabilitation movements is presented, encompassing various distance functions, dimensionality-reduction techniques, and movement models employed for this problem in prior studies. To the best of our knowledge, this is the first work that implements deep neural network for assessment of rehabilitation performance. Multiple deep network architectures are repurposed for the task in hand and are validated on a dataset of rehabilitation exercises.
Conventional approaches for mathematical modeling and representation of human movements are broadly classified into two categories: top-down approaches that introduce latent states for describing the temporal dynamics of the movements, and bottom-up approaches that employ local features for representing the movements. Commonly used methods in the first category include Kalman filters @cite_0 , hidden Markov models @cite_3 , and Gaussian mixture models @cite_36 . The main shortcomings of these methods originate from employing linear models for the transitions among the latent states (as in Kalman filters), or from adopting simple internal structures of the latent states (typical for hidden Markov models). The approaches based on extracting local features employ predefined criteria for identifying key points @cite_13 or a collection of statistics of the movements (e.g., mean, standard deviation, mode, median) @cite_22 . Such local features are typically motion-specific, which limits the ability to efficiently handle arbitrary spatio-temporal variations within movement data.
@cite_22 @cite_36 @cite_3 @cite_0 @cite_13
41309742 2151214862 2042919965 2099000909
This paper presents an approach to predict future motion of a moving object based on its past movement. This approach is capable of learning object movement in an open environment, which is one of the limitions in some prior works. The proposed approach exploits the similarities of short-term movement behaviors by modeling a trajectory as concatenation of short segments. These short segments are assumed to be noisy realizations of latent segments. The transitions between the underlying latent segments are assumed to follow a Markov model. This predictive model was applied to two real-world applications and yielded favorable performance on both tasks. Abstract We describe algorithms for recognizing human motion in monocular video sequences, based on discriminative conditional random fields (CRFs) and maximum entropy Markov models (MEMMs). Existing approaches to this problem typically use generative structures like the hidden Markov model (HMM). Therefore, they have to make simplifying, often unrealistic assumptions on the conditional independence of observations given the motion class labels and cannot accommodate rich overlapping features of the observation or long-term contextual dependencies among observations at multiple timesteps. This makes them prone to myopic failures in recognizing many human motions, because even the transition between simple human activities naturally has temporal segments of ambiguity and overlap. The correct interpretation of these sequences requires more holistic, contextual decisions, where the estimate of an activity at a particular timestep could be constrained by longer windows of observations, prior and even posterior to that timestep. This would not be computationally feasible with a HMM which requires the enumeration of a number of observation sequences exponential in the size of the context window. In this work we follow a different philosophy: instead of restrictively modeling the complex image generation process – the observation, we work with models that can unrestrictedly take it as an input, hence condition on it. Conditional models like the proposed CRFs seamlessly represent contextual dependencies and have computationally attractive properties: they support efficient, exact recognition using dynamic programming, and their parameters can be learned using convex optimization. We introduce conditional graphical models as complementary tools for human motion recognition and present an extensive set of experiments that show not only how these can successfully classify diverse human activities like walking, jumping, running, picking or dancing, but also how they can discriminate among subtle motion styles like normal walks and wander walks. We propose a unified model for human motion prior with multiple actions. Our model is generated from sample pose sequences of the multiple actions, each of which is recorded from real human motion. The sample sequences are connected to each other by synthesizing a variety of possible transitions among the different actions. For kinematically-realistic transitions, our model integrates nonlinear probabilistic latent modeling of the samples and interpolation-based synthesis of the transition paths. While naive interpolation makes unexpected poses, our model rejects them (1) by searching for smooth and short transition paths by employing the good properties of the observation and latent spaces and (2) by avoiding using samples that unexpectedly synthesize the nonsmooth interpolation. The effectiveness of the model is demonstrated with real data and its application to human pose tracking. In this paper a bottom-up approach for human behaviour understanding is presented, using a multi-camera system. The proposed methodology, given a training set of normal data only, classifies behaviour as normal or abnormal, using two different criteria of human behaviour abnormality (short-term behaviour and trajectory of a person). Within this system an one-class support vector machine decides short-term behaviour abnormality, while we propose a methodology that lets a continuous Hidden Markov Model function as an one-class classifier for trajectories. Furthermore, an approximation algorithm, referring to the Forward Backward procedure of the continuous Hidden Markov Model, is proposed to overcome numerical stability problems in the calculation of probability of emission for very long observations. It is also shown that multiple cameras through homography estimation provide more precise position of the person, leading to more robust system performance. Experiments in an indoor environment without uniform background demonstrate the good performance of the system.