1. An image processing apparatus comprising: a specific color determination unit that determines whether an input image is a specific color region that includes specific color in each predetermined size region, wherein the region of the input image include a plurality of pixels; a classifying unit that classifies the predetermined size regions of the input image to a first region, a second region or a third region, according to the determination by the specific color determination unit; and a specific region determination unit that determines a specific region to be subjected to predetermined image processing, according to the classifying by the classifying unit, wherein the classifying unit classifies, in order of a horizontal direction of the input image, a region which is the specific color region as a first region, a region which is not the specific color region as a third region, and a second region which is not the specific color region but upper region is a first region, wherein the classifying unit classifies a region which is classified as the second region as a first region, which is wedged by a region classified as a first region in the line which is being classified, before starting classifying processing of a bottom line which is being classified; wherein the specific region determination unit determines that the region classified as the first region is the specific region.