1. A method comprising: processing an input video sequence to determine a motion vector at a first level of accuracy; refining the motion vector at a second level of accuracy; selecting, with a selection element, a subset including less than all of candidate pixel locations at least partially based on whether a best candidate pixel location of the second level of accuracy is the same as a best candidate pixel location of the first level of accuracy; and determining the motion vector at a third level of accuracy based on the subset of candidate pixel locations, wherein when the best candidate location of the second level of accuracy is not the same as the best candidate pixel location of the first level of accuracy, selecting the subset of candidate pixel locations comprises selecting candidate pixel locations proximate to the best candidate pixel locations of the second level of accuracy that are between the best candidate pixel location of the second level of accuracy and the best candidate pixel location of the first level of accuracy.