1. An image pickup apparatus comprising: an image pickup device that has a plurality of pixels each of which is provided with a plurality of photoelectric conversion portions with respect to one micro lens thereof, such that light passing through a photographic optical system is pupil-divided; a holding unit configured to hold first defect information indicative of defect information of first image data generated by adding all signals output from the plurality of photoelectric conversion portions of each of the plurality of pixels and reading out a resulting added signal, and second defect information obtained by excluding information redundant with the first defect information from defect information of second image data generated by reading out a signal from one of the plurality of photoelectric conversion portions of each of the plurality of pixels or by adding signals output from part of the plurality of photoelectric conversion portions and reading out a resulting added signal; a designation unit configured to designate defective pixels included in the second image data based on the first defect information and the second defect information; and a correction unit configured to correct the second image data by correcting defective pixels designated by the designation unit.