We present a novel framework for the automatic discovery and recognition of motion primitives in videos of human activities. Given the 3D pose of a human in a video, human motion primitives are discovered by optimizing the motion flux', a quantity which captures the motion variation of a group of skelet al joints. A normalization of the primitives is proposed in order to make them invariant with respect to a subject anatomical variations and data sampling rate. The discovered primitives are unknown and unlabeled and are unsupervisedly collected into classes via a hierarchical non-parametric Bayes mixture model. Once classes are determined and labeled they are further analyzed for establishing models for recognizing discovered primitives. Each primitive model is defined by a set of learned parameters. Given new video data and given the estimated pose of the subject appearing on the video, the motion is segmented into primitives, which are recognized with a probability given according to the parameters of the learned models. Using our framework we build a publicly available dataset of human motion primitives, using sequences taken from well-known motion capture datasets. We expect that our framework, by providing an objective way for discovering and categorizing human motion, will be a useful tool in numerous research fields including video analysis, human inspired motion generation, learning by demonstration, intuitive human-robot interaction, and human behavior analysis.
The seemingly simple everyday actions of moving limb and body to accomplish a motor task or interact with the environment are incredibly complex. To reach for a target we first need to sense the target's position with respect to an external coordinate system; we then need to plan a limb trajectory which is executed by issuing an appropriate series of neural commands to the muscles. These, in turn, exert appropriate forces and torques on the joints leading to the desired movement of the arm. Here we review some of the earlier work as well as more recent studies on the control of human movement, focusing on behavioral and modeling studies dealing with task space and joint-space movement planning. At the task level, we describe studies investigating trajectory planning and inverse kinematics problems during point-to-point reaching movements as well as two-dimensional (2D) and three-dimensional (3D) drawing movements. We discuss models dealing with the two-thirds power law, particularly differential geometrical approaches dealing with the relation between path geometry and movement velocity. We also discuss optimization principles such as the minimum-jerk model and the isochrony principle for point-to-point and curved movements. We next deal with joint-space movement planning and generation, discussing the inverse kinematics problem and common solutions to the problems of kinematic redundancy. We address the question of which reference frames are used by the nervous system and review studies examining the employment of kinematic constraints such as Donders' and Listing's laws. We also discuss optimization approaches based on Riemannian geometry. One principle of motor coordination during human locomotion emerging from this body of work is the intersegmental law of coordination. However, the nature of the coordinate systems underlying motion planning remains of interest as they are related to the principles governing the control of human arm movements. Considerable evidence exists that the velocity of execution of handwriting and drawing movements depends on some global metric properties of the movement (size, linear extent etc.). Recent experiments have demonstrated that the instantaneous velocity also depends on the local curvature of the trajectory, that is, on the differential geometrical properties of the movement. In this paper we investigate further the role of the differential factors. Experiments are described in which drawing movements of simple geometrical forms and scribbles are performed either freely and extemporaneously, or in the presence of external constraints. It is shown that, at any time during the movement, the velocity component related to differential factors only depends on the value of the curvature of the trajectory at the same time (no dynamics). The relation can be described quantitatively as a specific Power Law and applies to all movements considered here, including those which are performed by following the edge of a template. The fact that the velocity of execution increases with the radius of curvature implies a built-in tendency of the motor control system to keep angular velocity relatively constant and qualifies the Isogony Principle proposed previously. The specific exponent of the Power Law suggests a possible interpretation of this empirical relation. In recent years different lines of evidence have led to the idea that motor actions and movements in both vertebrates and invertebrates are composed of elementary building blocks. The entire motor repertoire can be spanned by applying a well-defined set of operations and transformations to these primitives and by combining them in many different ways according to well-defined syntactic rules. Motor and movement primitives and modules might exist at the neural, dynamic and kinematic levels with complicated mapping among the elementary building blocks subserving these different levels of representation. Hence, while considerable progress has been made in recent years in unravelling the nature of these primitives, new experimental, computational and conceptual approaches are needed to further advance our understanding of motor compositionality. Humans interact with their environment through sensory information and motor actions. These interactions may be understood via the underlying geometry of both perception and action. While the motor space is typically considered by default to be Euclidean, persistent behavioral observations point to a different underlying geometric structure. These observed regularities include the “two-thirds power law”, which connects path curvature with velocity, and “local isochrony”, which prescribes the relation between movement time and its extent. Starting with these empirical observations, we have developed a mathematical framework based on differential geometry, Lie group theory and Cartan’s moving frame method for the analysis of human hand trajectories. We also use this method to identify possible motion primitives, i.e., elementary building blocks from which more complicated movements are constructed. We show that a natural geometric description of continuous repetitive hand trajectories is not Euclidean but equi-affine. Specifically, equi-affine velocity is piecewise constant along movement segments, and movement execution time for a given segment is proportional to its equi-affine arc-length. Using this mathematical framework, we then analyze experimentally recorded drawing movements. To examine movement segmentation and classification, the two fundamental equi-affine differential invariants—equi-affine arc-length and curvature are calculated for the recorded movements. We also discuss the possible role of conic sections, i.e., curves with constant equi-affine curvature, as motor primitives and focus in more detail on parabolas, the equi-affine geodesics. Finally, we explore possible schemes for the internal neural coding of motor commands by showing that the equi-affine framework is compatible with the common model of population coding of the hand velocity vector when combined with a simple assumption on its dynamics. We then discuss several alternative explanations for the role that the equi-affine metric may play in internal representations of motion perception and production.  Few computational models have addressed the spatiotemporal features of unconstrained three-dimensional (3D) arm motion. Empirical observations made on hand paths, speed profiles, and arm postures during point-to-point movements led to the assumption that hand path and arm posture are independent of movement speed, suggesting that the geometric and temporal properties of movements are decoupled. In this study, we present a computational model of 3D movements for an arm with four degrees of freedom based on the assumption that optimization principles are separately applied at the geometric and temporal levels of control. Geometric properties (path and posture) are defined in terms of geodesic paths with respect to the kinetic energy metric in the Riemannian configuration space. Accordingly, a geodesic path can be generated with less muscular effort than on any other, nongeodesic path, because the sum of all configuration-speed-dependent torques vanishes. The temporal properties of the movement (speed) are determined in task space by minimizing the squared jerk along the selected end-effector path. The integration of both planning levels into a single spatiotemporal representation simplifies the control of arm dynamics along geodesic paths and results in movements with near minimal torque change and minimal peak value of kinetic energy. Thus, the application of Riemannian geometry allows for a reconciliation of computational models previously proposed for the description of arm movements. We suggest that geodesics are an emergent property of the motor system through the exploration of dynamical space. Our data validated the predictions for joint trajectories, hand paths, final postures, speed profiles, and driving torques. Trajectory and kinematics of drawing movements are mutually constrained by functional relationships that reduce the degrees of freedom of the hand-arm system. Previous investigations of these relationships are extended here by considering their development in children between 5 and 12 years of age. Performances in a simple motor task—the continuous tracing of elliptic trajectories—demonstrate that both the phenomenon of isochrony (increase of the average movement velocity with the linear extent of the trajectory) and the so-called two-thirds power law (relation between tangential velocity and curvature) are qualitatively present already at the age of 5. The quantitative aspects of these regularities evolve with age, however, and steady-state adult performance is not attained even by the oldest children. The power-law formalism developed in previous reports is generalized to encompass these developmental aspects of the control of movement. Two general frameworks are currently available to conceptualize the motor-control problem. Broadly, the two frameworks differ in the answer that they give to the question "Where do form and structure come from?" According to the motor-program view (e.g., Schmidt, 1988), form and structure in a movement come from a central abstract representation of the intended result. Thus, making a movement entails a detailed mapping of this representation into an appropriate motor plan. The competing task-dynamic view (e.g., Kugler, 1989; Saltzman & Kelso, 1987) emphasizes instead the global morphogenetic power of the nonlinearities of the motor system. These two views need not be mutually exclusive. In fact, different classes of movements may require different conceptual frameworks for their analysis. At any rate (at least in the case of drawing movements considered here), an internal representation of the intended trajectory and kinematics must be available before execution. Consequently, much effort is being devoted to understanding how this internal model is translated into a set of motor commands. In this respect, one basic conceptual difficulty is because the geometry of the arm-forearm-hand system affords a large number of degrees of freedom. Provided that there are no constraints in the workspace, the only trajectory that matters in many manual 