1. An image processing apparatus that performs tracking processing in frame images that constitute a movie by detecting an object existing in a tracking target region set in a first image, in a second image, the image processing apparatus comprising: a setting unit configured to set the tracking target region in the first image; a tracking unit configured to detect in the second image a tracking target candidate region which has a highest correlation with the tracking target region set by the setting unit; a determination unit configured to calculate a difference between an evaluation value acquired in the tracking target candidate region and an evaluation value acquired in a peripheral region of the tracking target candidate region; and a control unit configured to update the tracking target region with the tracking target candidate region if the difference is equal to or more than a threshold value and to reset the tracking target region if the difference is less than the threshold value.