1. A method comprising: receiving a current frame to encode; analyzing the current frame to determine a primary feature for the frame and a secondary feature and a tertiary feature for a current macroblock in the frame, wherein the secondary feature is part of the primary feature and is based on at least a portion of an object found in the current frame, wherein the tertiary feature is part of the secondary feature and is based on at least a portion of an object found the in the current frame, wherein the primary feature is a first level in a hierarchy of features of the object, the secondary feature is a second level in the hierarchy of features of the object, and the tertiary feature is a third level in the hierarchy of features of the object; determining, based on the tertiary feature, a subset of reference frames from a storage of reference frames in memory, the reference frames classified based on a plurality of features, wherein a total number of the subset of reference frames is less than a total number of reference frames; performing a first motion search using a first block of data in the subset of reference frames, wherein a first difference value between the current macroblock and the first block of data is calculated; performing a second motion search using a second block of data in the subset of reference frames, wherein a second difference value between the current macroblock and the second block of data is calculated; classifying the first block of data as a reference block if the first difference value is less than the second difference value; classifying the second block of data as the reference block if the second difference value is less than the first difference value; and encoding the current macro block using the reference block.