1. An image pickup apparatus comprising: an image pickup device including a plurality of pixels two-dimensionally arrayed at a predetermined pixel pitch and configured to receive a light beam, pick up an image, and acquire the image; a pixel shift circuit configured to perform a pixel shift for changing a pixel shift position, which is a relative position in a direction of the two-dimensional array of the image pickup device and the light beam received by the image pickup device; a control circuit configured to cause the pixel shift circuit to perform pixel shifts to a plurality of different pixel shift positions and cause the image pickup device to perform the image pickup in each of the pixel shift positions and acquire a plurality of images; and a detection circuit configured to detect whether a defective image inappropriate as an image used to compose a high-quality image occurs while the predetermined number of images are acquired, wherein the control circuit controls the pixel shift circuit to perform the pixel shift in pixel shift order in which an image higher in image quality than one image acquired from the image pickup device can be composed even when, partway in the image pickup of a predetermined number of the images necessary for composing an image having predetermined high image quality, image processing is performed on the basis of two or more images less than the predetermined number, and when the occurrence of the defective image is detected by the detection circuit while the predetermined number of images are acquired, the control circuit performs control different from control performed when the occurrence of the defective image is not detected.