1. A tracking apparatus comprising: an image data acquisition unit configured to acquire image data per frame; a tracking process unit configured to detect a candidate position of a tracking target in each image data acquired per frame; a contrast information acquisition unit configured to acquire contrast information at the candidate position; a peripheral contrast information acquisition unit configured to acquire peripheral contrast information at peripheral contrast information acquisition areas around the candidate position; a contrast information similarity evaluation unit configured to evaluate a difference between contrast information at a tracking position of the tracking target decided in a past frame and contrast information at the candidate position in a current frame to determine whether the candidate position is a tracking position of the tracking target in the current frame or whether the tracking target is lost; when the difference is lower than a threshold value, a control unit decides that the candidate position is the tracking position of the tracking target in the current frame; and when the difference is higher than the threshold value, a tracking position correction determination causes the contrast information similarity evaluation unit to evaluate a similarity between the contrast information at the tracking position of the tracking target decided in the past frame and the peripheral contrast information of the peripheral contrast information acquisition areas, and decides that a peripheral contrast information acquisition area having the highest similarity among the peripheral contrast information acquisition areas is the tracking position of the tracking target in the current frame.