1. An image processing device which corrects an image with a degraded image quality due to unnecessary diffracted light generated in an optical system including a diffractive optical element, said image processing device comprising: an evaluation area setting unit configured to detect a saturated pixel having a saturated luminance value, from among pixels included in the image and to set an area in the image including the detected saturated pixel as an evaluation area; a pixel count calculating unit configured to calculate a pixel count for each of luminance values of pixels included in the set evaluation area; a first luminance value determining unit configured to determine a first luminance value indicating a maximum luminance of an image of the unnecessary diffracted light, based on a transition of calculated pixel counts arranged in order of luminance value; a second luminance value determining unit configured to determine a second luminance value which is a luminance value corresponding to a background of the image; and a correction unit configured to correct a luminance value of a pixel of luminance lower than luminance of the first luminance value and higher than luminance of the second luminance value from among the pixels included in the evaluation area so as to lower the luminance of the pixel.