1. An image processing apparatus comprising: image acquiring means configured to acquire an input image; and image restoration processing means configured to generate a restored image by calculating the input image and an image restoration filter, the image restoration filter being based on a transfer function of an image pickup system that is used to form an object image as the input image, wherein the image restoration filter performs restoration so that when an object is a white point light source, a difference between spectra of two color components in the restored image is made smaller than a difference between spectra of the two color components in the input image.