1. An object tracking device that receives an image signal having a plurality of frame images and tracks a specific object in the image signal, comprising: a detection unit configured to detect, in a current frame image, an analogous area having a feature value most approximate to that of a reference area, the reference area being indicative of and covering an area where an image of the object is formed in a portion of a preceding frame image preceding the current frame image; a small area-setting unit configured to set a predetermined number of small areas in the reference area; a motion vector detection unit configured to detect a motion vector of the object in each of the small areas; a shape change determination unit configured to determine a change of the object, related to a change in orientation of the object, according to the motion vector to thereby obtain shape change information; an area correction unit configured to correct a location and a size of the analogous area according to the shape change information to thereby correct the analogous area to a corrected analogous area in the current frame image; and a tracking unit configured to track the object using the corrected analogous area.