1. A three-dimensional image processing apparatus which is capable of acquiring a distance image that includes height information of an inspection target and also performing image processing based on the distance image, the apparatus comprising: a light projecting part for projecting incident light as structured illumination of a predetermined fringe-like projection pattern from an oblique direction with respect to an optical axis of a below-described image capturing part; the image capturing part for acquiring reflected light that is projected by the light projecting part and reflected on an inspection target, to capture a plurality of pattern projected images; a distance image generating part capable of generating a distance image based on the plurality of images captured in the image capturing part; a pattern generating part for generating a first projection pattern and a second projection pattern whose fringe direction is different from that of the first projection pattern as a plurality of projection patterns obtained by changing a fringe direction of the projection pattern; and an incorrect-height determining part for making comparison in height information of a corresponding portion of the inspection target between a first distance image, generated in the distance image generating part based on a first pattern projected image obtained by projecting from the light-projecting part the first projection pattern generated in the pattern generating part and capturing an image of the first projection pattern in the image capturing part, and a second distance image, generated based on a second pattern projected image obtained by the second projection pattern projected from the common light projecting part, to determine height information of a portion where a difference not smaller than a predetermined value occurs as incorrect.