Hand motion capture has been an active research topic, following the success of full-body pose tracking. Despite similarities, hand tracking proves to be more challenging, characterized by a higher dimensionality, severe occlusions and self-similarity between fingers. For this reason, most approaches rely on strong assumptions, like hands in isolation or expensive multi-camera systems, that limit practical use. In this work, we propose a framework for hand tracking that can capture the motion of two interacting hands using only a single, inexpensive RGB-D camera. Our approach combines a generative model with collision detection and discriminatively learned salient points. We quantitatively evaluate our approach on 14 new sequences with challenging interactions.
The study of hand motion tracking has its roots in the @math s @cite_20 @cite_46 . Although the problem can be simplified by means of data-gloves @cite_35 , color-gloves @cite_25 , markers @cite_6 or wearable sensors @cite_26 , the ideal solution pursued is the unintrusive, marker-less capture of hand motion. Even until recently the study was mainly confined to the case of a single isolated hand @cite_3 @cite_41 @cite_9 @cite_19 @cite_24 @cite_1 @cite_18 . However, in pursuit of more realistic scenarios, research effort was directed towards the case of a hand interacting with an object @cite_27 @cite_31 @cite_0 , two hands interacting with each other @cite_8 @cite_21 and with an additional object @cite_8 . Multiple objects can be tracked by means of hand tracking and physical forces modeling @cite_23 .
@cite_35 @cite_41 @cite_3 @cite_20 @cite_18 @cite_8 @cite_21 @cite_23 @cite_46 @cite_26 @cite_6 @cite_19 @cite_27 @cite_25 @cite_9 @cite_1 @cite_24 @cite_0 @cite_31
276900368 2564823405 2227547437 2552247836
Hand motion capture has been an active research topic, following the success of full-body pose tracking. Despite similarities, hand tracking proves to be more challenging, characterized by a higher dimensionality, severe occlusions and self-similarity between fingers. For this reason, most approaches rely on strong assumptions, like hands in isolation or expensive multi-camera systems, that limit practical use. In this work, we propose a framework for hand tracking that can capture the motion of two interacting hands using only a single, inexpensive RGB-D camera. Our approach combines a generative model with collision detection and discriminatively learned salient points. We quantitatively evaluate our approach on 14 new sequences with challenging interactions. Abstract This paper describes the development of a remote handling control of an anthropomorphic robotic hand, the SCHUNK S5FH, using the human hand as master by measuring its motion with OptiTrack Technology. The goal of this work is to enhance manipulation studies on the human hand and to instantly transfer those studies on robotic hands. A preliminary study on methods and devices used for fingers tracking led to the choice of a simplified kinematic model of the human hand on the basis of the available motion tracking system. Using the same criteria, the analysis of protocols for markers allocation led to define the number and a method for their arrangement on the fingers and palm. In order to overcome the limitation of the Motion Capture System, a method for identification and labeling has been developed according to their anatomical arrangement. Afterwards, the tracking is performed using the constraints between marker positions on the kinematic chain of the hand and a dynamic labeling algorithm robust with respect to noise, outliers and loss of markers. The validation is performed using the right hand of different subjects and considering different tasks involving flexion extension and abduction adduction of fingers and thumb opposition. For testing and validation, preliminary studies on synergies for manipulation tasks such as screwing a cup, has been conducted on the human hand and transferred on the robotic hand. Hand motion capture is a popular research field, recently gaining more attention due to the ubiquity of RGB-D sensors. However, even most recent approaches focus on the case of a single isolated hand. In this work, we focus on hands that interact with other hands or objects and present a framework that successfully captures motion in such interaction scenarios for both rigid and articulated objects. Our framework combines a generative model with discriminatively trained salient points to achieve a low tracking error and with collision detection and physics simulation to achieve physically plausible estimates even in case of occlusions and missing visual data. Since all components are unified in a single objective function which is almost everywhere differentiable, it can be optimized with standard optimization techniques. Our approach works for monocular RGB-D sequences as well as setups with multiple synchronized RGB cameras. For a qualitative and quantitative evaluation, we captured 29 sequences with a large variety of interactions and up to 150 degrees of freedom. Modern systems for real-time hand tracking rely on a combination of discriminative and generative approaches to robustly recover hand poses. Generative approaches require the specification of a geometric model. In this paper, we propose a the use of sphere-meshes as a novel geometric representation for real-time generative hand tracking. How tightly this model fits a specific user heavily affects tracking precision. We derive an optimization to non-rigidly deform a template model to fit the user data in a number of poses. This optimization jointly captures the user's static and dynamic hand geometry, thus facilitating high-precision registration. At the same time, the limited number of primitives in the tracking template allows us to retain excellent computational performance. We confirm this by embedding our models in an open source real-time registration algorithm to obtain a tracker steadily running at 60Hz. We demonstrate the effectiveness of our solution by qualitatively and quantitatively evaluating tracking precision on a variety of complex motions. We show that the improved tracking accuracy at high frame-rate enables stable tracking of extended and complex motion sequences without the need for per-frame re-initialization. To enable further research in the area of high-precision hand tracking, we publicly release source code and evaluation datasets.