1. An image processing apparatus comprising: a primary restored image generating module configured to generate a primary restored image by performing image restoration processing on an input image having a plurality of color components; a restoration component generating module configured to generate difference information for each of the plurality of color components by calculating a difference between the input image and the primary restored image; and a secondary restored image generating module configured to generate a secondary restored image using the difference information, wherein the secondary restored image generating module combines a color-combination restoration component based on the difference information for the plurality of color components with one color component of the input image.