FullHand: Markerless Skeleton based Tracking for Free Hand Interaction
This paper advances a novel markerless hand tracking method for interactive applications. FullHand uses input from RGB and depth cameras in a desktop setting. It combines, in a voting scheme, a discriminative, part based pose retrieval with a generative pose estimation method based on local optimization. We develop this approach to enable: (1) capturing hand articulations with high number of degrees of freedom, including the motion of all fingers, (2) sufficient precision, shown in a dataset of user generated gestures, and (3) a high framerate of 50 fps for one hand. We discuss the design of free hand interactions with the tracker and present several demonstrations ranging from simple (few DOFs) to complex (finger individuation plus global hand motion) including mouse operation, a first person shooter and virtual globe navigation. A user study on the latter shows that free hand interactions implemented for the tracker can equal mouse based interactions in user