1. An image processing apparatus comprising: a specifying unit configured to specify pixels other than a white pixel which are included in a window, the window including a target pixel and pixels in a vicinity of the target pixel in a scanned image; an averaging unit configured to average color values of the specified pixels other than the white pixel; and a determination unit configured to determine whether the target pixel is a color pixel based on a color value obtained by the averaging.