1. An image processing device comprising: an obtaining unit configured to obtain a plurality of color images each of which is captured, from a viewpoint different from each other, through a color filter array comprising a plurality of color filters corresponding to a plurality of colors, wherein each pixel of the plurality of color images misses information of at least one of the plurality of colors; and a generating unit configured to generate, by combining the plurality of color images before performing an interpolation for the missing color information, and by performing an interpolation for color information on a combined image which is a result of the combination of the plurality of color images, a composite image which corresponds to an arbitrary focus position and comprises at least one pixel having information of all of the plurality of colors, wherein the combined image includes at least one missing pixel having information of at least one of the plurality of colors and missing color information of at least one of the plurality of colors, and the generating unit performs interpolation for the missing color information on the missing pixel.