1. An image processing device comprising: an image sensor configured to capture a plurality of pupil-divided images obtained from light fluxes passing through different pupil regions of an imaging optical system; and a selection unit configured to select an image to be used for image processing from the plurality of pupil-divided images, according to a position of an imaging area in the image sensor, wherein the plurality of pupil-divided images includes at least a first pupil-divided image obtained from a first pixel in the image sensor, and a second pupil-divided image obtained from a second pixel in the image sensor, the imaging area in the image sensor is divided into a plurality of areas including at least a first area and a second area, regarding the first area, the selection unit selects the first pupil-divided image as an image which involves a first exposure amount and is used for the image processing, and selects another pupil-divided image, other than the first pupil-divided image, as an image which involves a second exposure amount less than the first exposure amount and is used for the image processing, and regarding the second area, the selection unit selects the second pupil-divided image as an image which involves the first exposure amount and is used for the image processing, and selects another pupil-divided image, other than the second pupil-divided image, as an image which involves the second exposure amount and is used for the image processing.