1. A subject tracking apparatus which tracks a specific subject in an image over a plurality of images input thereto in time series, comprising: a computation circuit configured to obtain, with respect to each of a plurality of reference images registered in registration unit, a maximum degree of correlation of the input image and a region which is included in the input image and has the maximum degree of correlation; and a first determination circuit configured to determine a region of the specific subject in the input image from the regions obtained by said computation circuit, wherein the plurality of reference images include a first reference image which is based on a region specified as the region of the specific subject in a first input image among the plurality of input images, and a second reference image which is based on a region determined as the region of the specific subject by said first determination circuit in a second input image, input to the subject tracking apparatus later than the first input image, among the plurality of input images, and said first determination circuit determines, without relying on a region with a maximum degree of correlation with the second reference image, that a region with a maximum degree of correlation with the first reference image is the region of the specific subject in the input image if the maximum degree of correlation obtained with respect to the first reference image by said computation circuit is not less than a predetermined threshold, and determines, based on the maximum degree of correlation obtained with respect of the first reference image and that obtained with respect to the second reference image, that either a region with a maximum degree of correlation with the first reference image or a region with a maximum degree of correlation with the second reference image is the region of the specific subject in the input image if the maximum degree of correlation obtained with respect to the first reference image by said computation circuit is less than the predetermined threshold.