1. A subject tracking apparatus that tracks a subject included in images that are sequentially supplied, the apparatus comprising: one or more processors; and a memory storing instructions which, when the instructions are executed by the one or more processors, cause the subject tracking apparatus to function as units comprising: a first registering unit configured to register a partial area indicative of the subject in one image of the supplied images as a template; a first matching unit configured to estimate a subject area by collating a partial area in a newly supplied image with the template registered by the first registering unit; a second registering unit configured to generate a histogram based on pixel values of a partial area indicative of the subject in one image of the supplied images and register the generated histogram; a second matching unit configured to estimate a subject area by collating a histogram based on pixel values of a partial area in a newly supplied image with the histogram registered by the second registering unit; and a tracking area determination unit configured to determine a tracking area based on either of a first area that is the subject area estimated by the first matching unit or a second area that is the subject area estimated by the second matching unit, wherein the partial area of which the second registering unit generates the histogram based on the pixel values is larger than the partial area registered as the template by the first registering unit.