1. An image processing apparatus comprising: an area extraction unit configured to extract a reference area from image data; an extraction unit configured to extract a plurality of small sections where pixels included in the reference area are arranged so as to be symmetrical with respect to a center pixel of the reference area in the horizontal and vertical directions, and point-symmetrical with respect to the center pixel, while overlap of pixels is allowed between each of the small sections; a pattern determination unit configured to determine whether or not image data included in the reference area matches a predetermined pattern; a characteristic determination unit configured to determine a characteristic of image data of the reference area based on a number of characteristics of the plurality of small sections extracted by said extraction unit; a judgment unit configured to judge whether or not to perform smoothing on the center pixel of the reference area based on results of determination determined by said pattern determination unit and said characteristic determination unit; and a correction processing unit configured to perform smoothing on the center pixel that has been judged by said judgment unit that smoothing is to be performed.