1. An image pickup apparatus comprising: an image pickup element that includes a plurality of image pickup pixels each configured to generate an image of an object by performing a photoelectric conversion for light from an image pickup lens, and a plurality of focus detecting pixels each configured to receive light that passes part of an exit pupil of the image pickup lens; a detector configured to detect a phase difference between two images of the object on a secondary imaging plane; and a controller configured to provide focusing based on a defocus amount obtained from a detection result of the detector for a first focus detecting point, and set a correcting value used to correct a defocus amount obtained from a next detection result of the detector so that a difference can be reduced between the defocus amount obtained from the detection result of the detector for the first focus detecting point and a defocus amount obtained from the focus detecting pixels for the first focus detecting point of a captured image of the object that has been captured.