1. An image processing apparatus that corrects a depth image representing information about a depth to a subject in a scene, which is the same scene of a plurality of images obtained by photographing the same subject from different viewpoints, the image processing apparatus comprising: a unit configured to determine a pixel of interest in a first image that is taken to be a reference among the plurality of images and peripheral pixels existing within a predetermined region according to the pixel of interest, the peripheral pixels being pixels for which a weight is to be derived; an evaluation value derivation unit configured to derive a matching evaluation value between the plurality of images for the respective peripheral pixels; a weight determination unit configured to determine a weight for the respective peripheral pixels in the correction based on the evaluation value; and a correction unit configured to correct a pixel value of the pixel of interest in the depth image by using the weight and a pixel value of the peripheral pixels, wherein the evaluation value derivation unit specifies, for each peripheral pixel, a pixel corresponding to the pixel of interest in a second image among the plurality of images, the second image being different from the first image, by using a depth of the respective peripheral pixels in the depth image and derives the matching evaluation value of each peripheral pixel based on the pixel value of the pixel of interest and the pixel value of the specified corresponding pixel.