1. An image processing apparatus comprising: a memory configured to store information of an optical transfer function or a point spread function of an image pickup optical system for at least one of capturing conditions; and an image processor configured: to generate secondary and higher components of a phase of an optical transfer function or a shape component of a point spread function corresponding to a capturing condition of an image captured through the image pickup optical system through an interpolation based on at least two optical transfer functions or point spread functions which correspond to different capturing conditions and are derived from the information stored in the memory, while center-of-gravity positions or maximum intensity positions are accorded with each other or differential root-mean-square values of the point spread functions are minimum, and to restore the image utilizing an optical transfer function derived from the optical transfer function that has been generated through the interpolation or an optical transfer function derived from the point spread function that has been generated through the interpolation.