1. An image pickup apparatus comprising: a first image pickup element that includes a plurality of image pickup pixels each configured to generate an image of an object by performing a photoelectric conversion for light from an image pickup lens, wherein the first image pickup element is configured to be located on a secondary imaging plane; a second image pickup element that includes a plurality of image pickup pixels each configured to generate an image of an object by performing a photoelectric conversion for light from an image pickup lens, wherein the second image pickup element is configured to be located on a primary imaging plane, wherein the second pickup element is configured to be able to output signal for recording image; a first detector configured to detect a defocus amount related to a phase difference between two images of the object from the first image pickup element; a second detector configured to detect a defocus amount based on a phase difference for two images of the object recorded from the second image pickup element; and a controller configured to provide focusing based on a defocus amount obtained from a detection result of the first detector, and set a correcting value used to correct a defocus amount obtained from a detection result of the first detector so that a difference can be reduced between the defocus amount obtained from the detection result of the first detector and a defocus amount obtained from the detection result of the second detector, wherein the correcting value is acquired from information according to the defocus amount detected by the second detector by using signal for recording image.