1. An image processing apparatus comprising: at least one processor; a memory that includes instructions for the at least one processor to, by being executed by the at least one processor; set a search area based on spatial information of a subject, the spatial information being information of an area in which the subject in an image is predicted to be present; set a first area inside the search area but not outside the search area; set a second area outside the first area; calculate a first feature amount of the first area; calculate a second feature amount of the second area, the second feature amount being a feature amount of the same type as the first feature amount; and calculate a degree of visual saliency of the subject in the first area based on a difference between the first feature amount and the second feature amount, wherein in a case that a plurality of search areas are set, the first area and the second area are set and the degree of visual saliency is calculated in each of the search areas, and a processing target area is determined from the first areas of the plurality of search areas based on the calculated degree of visual saliency.