1. An image processing device for obtaining a depth information image representing a distribution of depth information about an object on the basis of a first image and a second image that have different degrees of blurring, the first and second images being captured by an image sensor including a plurality of groups each including a plurality of pixels that obtain different color information about the object, the image processing device comprising: an extracting unit configured to extract signals, each corresponding to one of the plurality of pixels of each group, from the plurality of groups and generate a first extracted image and a second extracted image respectively corresponding to the first image and the second image; and a calculating unit configured to calculate the depth information image from a difference in degree of blurring between the first extracted image and the second extracted image, wherein the signals each corresponding to one of the plurality of pixels of each group and extracted from the plurality of groups are signals corresponding to pixels that obtain the same color information.