1. An image processing apparatus comprising: a storage device configured to store a background model in which a feature amount is associated with time information for each of a plurality of states at each position of an image to be a background; a memory configured to store a program; and a processor configured to execute the program to provide: an extraction unit configured to extract a feature amount for each position of an input video image; an input state determination unit configured to compare the feature amount in the input video image with the feature amount of each of the plurality of states in the background model, to determine the state similar to the input video image, and update the time information of the state that has been determined to be similar to the input video image; a foreground determination unit configured to determine a foreground area in the input video image based on the time information of the state that has been determined to be similar to the input video image by the input state determination unit; a subject detection unit configured to detect a predetermined subject from the foreground area; and an updating unit configured to update the time information of the state in the background model based on a detection result obtained by the subject detection unit.