1. An image processing apparatus for suppressing color fringing of an image defined by a plurality of pixels, comprising: a reference gradient detecting unit adapted to detect a gradient from an image signal of a luminance component or one of color components of a pixel, which is used as a reference image signal, and output a reference gradient signal; a suppression target gradient detecting unit adapted to detect gradients of image signals of color components different from the reference image signal of the pixel, which are used as suppression target image signals, and output suppression target gradient signals; a suppression coefficient computing unit adapted to compute color fringing suppression coefficients of the respective suppression target image signals of the pixel, based on the reference gradient signal and the suppression target gradient signals; and a combining unit adapted to combine each of the suppression target image signals of the pixel with the reference image signal by weighting the reference image signal based on the color fringing suppression coefficients, and output the combined signals as image signals of the color components in which the color fringing is suppressed, wherein said suppression coefficient computing unit defines a plurality of regions in accordance with magnitude relationships between values of the reference gradient signal and the suppression target gradient signals, allocates a function in advance to each of the regions and computes the color fringing suppression coefficients of the respective suppression target image signals of the pixel by classifying the respective suppression target image signals into one of the plurality of regions and using the respective function allocated to that region to compute the color fringing suppression coefficient.