BACKGROUND

Hand tracking refers to the process of detecting the poses of a user's hand. For example, cameras can be set up to capture images of the user's hand. The images may be processed to determine a hand pose, which may be used as an input to for a computing device. The speed and accuracy of the programmatic hand pose analysis from images depends on various factors, such as the ability to recognize variations in hand size and shape across different users. It is desirable to provide highly accurate hand model calibration and hand pose identification from images, while reducing computational time and hardware requirements.

SUMMARY

Techniques for generating a user hand shape model for hand tracking are discussed herein. A user hand shape model customized for a user is generated as a set of principal component hand shape models, such as a weighted combination of ten principal component hand shape models. Each principal component hand shape model may control a linearly uncorrelated parameter of user hand shape. The use of a limited number of principal component hand shape models reduces determination of user hand shape to a smaller number of variables, and thus provides for a fast calibration of the user hand shape model.

The system may include a hand tracking unit that receives depth image data including a user's hand from a single depth camera. The hand tracking unit includes a neural network that, using the depth image data, generates probable locations for various hand features on a user's hand, which is used to determine a hand pose of the user's hand. The neural network may be a convolutional neural network. The hand tracking unit performs an optimization that relates the hand pose to a user hand shape model defining the shape of the user's hand. The limited number of principal component hand shape models provide for a limited number of variables for a rapid determination of hand shape. The user hand shape model can be used to facilitate subsequent determination of hand poses from depth image data.

Some embodiments may include a system, comprising a single depth camera and a hand tracking unit. The single depth camera generates single depth image data of a user's hand. The hand tracking unit receives the single depth image data of the user's hand from the single depth camera, and determines based on applying the single depth image data to a neural network model, heat maps indicating locations of hand features. A hand pose is determined based on the location of the hand features, and an input hand shape model is determined based on the hand pose. A set of principal component hand shape models is determined, and a user hand shape model is determined based on fitting the input hand shape model to the set of principal component hand shape models.

In some embodiments, the user hand shape model may include a combination of the set of principal component hand shape models. For example, the user hand shape model may be defined as a weighted combination of the principal component hand shape models.

Some embodiments may include a device including a hand tracking unit. The device may be a head-mounted display (HMD), or a console attached with a HMD. The hand tracking unit receives single depth image data of a user's hand from a single depth camera, and determines, based on applying the single depth image data to a neural network model, heat maps indicating locations of hand features. A hand pose is determined based on the location of the hand features, and an input hand shape model is determined based on the hand pose. A set of principal component hand shape models is determined, and a user hand shape model is determined based on fitting the input hand shape model to the set of principal component hand shape models.

