1. An image processing apparatus comprising: a reference surface generating unit that generates a reference surface indicating a reference value of each of a plurality of color elements of pixels constituting an intraluminal image at a pixel position of the each color element by performing a morphology process using pixel values of the color elements; and an abnormal area detecting unit that detects an abnormal area from the intraluminal image based on a difference for each of the color elements between the pixel value of each pixel and the reference surface.