1. An apparatus, comprising: a depth-sensing camera; a memory having, stored therein, computer program code; and one or more processing units operatively coupled to the memory and configured to execute instructions in the computer program code that cause the one or more processing units to: receive a depth map of a scene containing at least one human hand from the depth-sensing camera, the depth map comprising a matrix of pixels, each pixel having a depth value; extract, from the depth map, descriptors based on the depth values of the pixels in a plurality of patches distributed in respective positions over the at least one human hand; match the extracted descriptors to previously-stored descriptors using a background-invariant decision forest; estimate a position of at least one of the at least one human hand based, at least in part, on stored information associated with the matched descriptors; and track the position of the at least one of the at least one human hand, wherein the instructions to track comprise instructions to track bi-directionally along a z-axis of the scene.