1. A method comprising: under control of one or more computing devices configured with executable instructions, identifying a first image, a second image and a third image, the first image captured prior to the second image and the second image captured prior to the third image, wherein: a first motion flow value between the second image and the third image is below a predetermined threshold, and a second motion flow value between the first image and the second image is above a predetermined threshold; determining a hand shape represented in at least one of the second image and the third image based on a comparison of at least a portion of the second image and the third image; obtaining a first plurality of color values associated with the determined hand shape; generating a custom color spectrum for use in tracking hand movement based at least in part on the obtained first plurality of color values, wherein at least one of: a maximum color value of the custom color spectrum is different than a maximum color value of a predefined color spectrum; or a minimum color value of the custom color spectrum is different than a minimum color value of the predefined color spectrum; associating the custom color spectrum with a user whose hand shape is represented in at least one of the second image and the third image; obtaining a plurality of additional images subsequent to the third image; determining a second plurality of color values represented in the plurality of additional images that are between the maximum color value of the custom color spectrum and the minimum color value of the custom color spectrum; and tracking a movement of the user's hand based on the second plurality of color values represented in the plurality of additional images.