1. An object area tracking apparatus for detecting a target object area from a supplied image and tracking the detected object area, comprising: a first detection unit configured to detect a predetermined object area from the image on the basis of a first feature amount; a second detection unit configured to detect a predetermined object area from the image on the basis of a second feature amount different from the first feature amount; and a main object determination unit configured to obtain a priority for each of the objects by using a detection result of the first detection unit and a detection result of the second detection unit and determine a main object of a high priority from the objects, wherein for the object which is not detected by the first detection unit and is detected by the second detection unit, the main object determination unit changes the priority in accordance with a detection result of the past of the object in the first detection unit.