Continuous action recognition is more challenging than isolated recognition because classification and segmentation must be simultaneously carried out. We build on the well known dynamic time warping framework and devise a novel visual alignment technique, namely dynamic frame warping (DFW), which performs isolated recognition based on per-frame representation of videos, and on aligning a test sequence with a model sequence. Moreover, we propose two extensions which enable to perform recognition concomitant with segmentation, namely one-pass DFW and two-pass DFW. These two methods have their roots in the domain of continuous recognition of speech and, to the best of our knowledge, their extension to continuous visual action recognition has been overlooked. We test and illustrate the proposed techniques with a recently released dataset (RAVEL) and with two public-domain datasets widely used in action recognition (Hollywood-1 and Hollywood-2). We also compare the performances of the proposed isolated and continuous recognition algorithms with several recently published methods.
We address the problem of learning view-invariant 3D models of human motion from motion capture data, in order to recognize human actions from a monocular video sequence with arbitrary viewpoint. We propose a Spatio-Temporal Manifold (STM) model to analyze non-linear multivariate time series with latent spatial structure and apply it to recognize actions in the joint-trajectories space. Based on STM, a novel alignment algorithm Dynamic Manifold Warping (DMW) and a robust motion similarity metric are proposed for human action sequences, both in 2D and 3D. DMW extends previous works on spatio-temporal alignment by incorporating manifold learning. We evaluate and compare the approach to state-of-the-art methods on motion capture data and realistic videos. Experimental results demonstrate the effectiveness of our approach, which yields visually appealing alignment results, produces higher action recognition accuracy, and can recognize actions from arbitrary views with partial occlusion. 2D Tracking.- Understanding Human Motion: A Historic Review.- The Role of Manifold Learning in Human Motion Analysis.- Recognition of Action as a Bayesian Parameter Estimation Problem over Time.- The William Harvey Code: Mathematical Analysis of Optical Flow Computation for Cardiac Motion.- Detection and Tracking of Humans in Single View Sequences Using 2D Articulated Model.- Learning.- Combining Discrete and Continuous 3D Trackers.- Graphical Models for Human Motion Modelling.- 3D Human Motion Analysis in Monocular Video: Techniques and Challenges.- Spatially and Temporally Segmenting Movement to Recognize Actions.- Topologically Constrained Isometric Embedding.- 2D-3D Tracking.- Contours, Optic Flow, and Prior Knowledge: Cues for Capturing 3D Human Motion in Videos.- Tracking Clothed People.- An Introduction to Interacting Simulated Annealing.- Motion Capture for Interaction Environments.- Markerless Motion Capture for Biomechanical Applications.- Biomechanics and Applications.- Qualitative and Quantitative Aspects of Movement: The Discrepancy Between Clinical Gait Analysis and Activities of Daily Life.- Optimization of Human Motion Exemplified with Handbiking by Means of Motion Analysis and Musculoskelet al Models.- Imitation Learning and Transferring of Human Movement and Hand Grasping to Adapt to Environment Changes.- Accurate and Model-free Pose Estimation of Crash Test Dummies.- Modelling and Animation.- A Relational Approach to Content-based Analysis of Motion Capture Data.- The Representation of Rigid Body Motions in the Conformal Model of Geometric Algebra.- Video-based Capturing and Rendering of People.- Interacting Deformable Objects.- From Performance Theory to Character Animation Tools. Several researchers have addressed the problem of human action recognition using a variety of algorithms. An underlying assump- tion in most of these algorithms is that action boundaries are already known in a test video sequence. In this paper, we propose a fast method for continuous human action recognition in a video sequence. We pro- pose the use of a low dimensional feature vector which consists of (a) the projections of the width profile of the actor on to a Discrete Co- sine Transform (DCT) basis and (b) simple spatio-temporal features. We use an earlier proposed average-template with multiple features for modelling human actions and combine it with One-pass Dynamic Pro- graming (DP) algorithm for continuous action recognition. This model accounts for intra-class variability in the way an action is performed. Furthermore, we demonstrate a way to perform noise robust recognition by creating a noise match condition between the train and the test data. The effectiveness of our method is demonstrated by conducting experi- ments on the IXMAS dataset of persons performing various actions and an outdoor Action database collected by us. Alignment of time series is an important problem to solve in many scientific disciplines. In particular, temporal alignment of two or more subjects performing similar activities is a challenging problem due to the large temporal scale difference between human actions as well as the inter intra subject variability. In this paper we present canonical time warping (CTW), an extension of canonical correlation analysis (CCA) for spatio-temporal alignment of human motion between two subjects. CTW extends previous work on CCA in two ways: (i) it combines CCA with dynamic time warping (DTW), and (ii) it extends CCA by allowing local spatial deformations. We show CTW's effectiveness in three experiments: alignment of synthetic data, alignment of motion capture data of two subjects performing similar actions, and alignment of similar facial expressions made by two people. Our results demonstrate that CTW provides both visually and qualitatively better alignment than state-of-the-art techniques based on DTW. Within the context of hand gesture recognition, spatiotemporal gesture segmentation is the task of determining, in a video sequence, where the gesturing hand is located and when the gesture starts and ends. Existing gesture recognition methods typically assume either known spatial segmentation or known temporal segmentation, or both. This paper introduces a unified framework for simultaneously performing spatial segmentation, temporal segmentation, and recognition. In the proposed framework, information flows both bottom-up and top-down. A gesture can be recognized even when the hand location is highly ambiguous and when information about when the gesture begins and ends is unavailable. Thus, the method can be applied to continuous image streams where gestures are performed in front of moving, cluttered backgrounds. The proposed method consists of three novel contributions: a spatiotemporal matching algorithm that can accommodate multiple candidate hand detections in every frame, a classifier-based pruning framework that enables accurate and early rejection of poor matches to gesture models, and a subgesture reasoning algorithm that learns which gesture models can falsely match parts of other longer gestures. The performance of the approach is evaluated on two challenging applications: recognition of hand-signed digits gestured by users wearing short-sleeved shirts, in front of a cluttered background, and retrieval of occurrences of signs of interest in a video database containing continuous, unsegmented signing in American sign language (ASL).