1. An image pickup apparatus comprising: an image-pickup lens having an aperture diaphragm; an image-pickup element that has a color filter which is periodically allocated a plurality of colors in units of a predetermined unit array and that generates picked-up image data including pixel data of a plurality of colors using received light, the color filter being adjacent to a light receiving surface; a microlens array unit that is arranged on an image forming surface of the image-pickup lens and whose microlenses are each allocated a plurality of pixels of the image-pickup element; and an image processing unit that performs image processing on the picked-up image data generated by the image-pickup element, wherein the image processing unit includes a parallax component image generation unit that generates a plurality of parallax component images, each of which is generated by extracting pieces of pixel data that are arranged at the same position in pixel areas corresponding to the microlenses in the picked-up image data and combining the pieces of extracted pixel data that are arranged at the same position, and an interpolation processing unit that performs color interpolation processing on each of the parallax component images, and wherein the color arrangement of the unit array of the color filter is the same as the color arrangement of the unit array in pixel data constituting each of the parallax component images, the unit array containing a plurality of colors.