1. A three-dimensional shape detecting device comprising a projection unit which projects pattern light, an image capturing unit which captures a pattern light projection image of a subject on which the pattern light is projected, and a three-dimensional shape calculation unit which calculates three-dimensional shape of the subject based on a locus of the pattern light extracted from the pattern light projection image captured by the image capturing unit, wherein the three-dimensional shape detecting device further comprises: a first area setting means unit which sets a first area, including a part of the locus of the pattern light extending in an X-axis direction of the pattern light projection image, in the pattern light projection image; a first detecting unit which detects a plurality of pixels forming the part of the locus of the pattern light in the first area set by the first area setting unit; an approximated curve calculation unit which calculates an approximated curve regarding the locus of the pattern light based on the plurality of pixels detected by the first detecting unit; and a second area setting unit which sets a second area, for detecting pixels forming remaining part of the locus of the pattern light in the pattern light projection image, based on the approximated curve calculated by the approximated curve calculation unit, wherein the locus of the pattern light is extracted based on the plurality of pixels detected by the first detecting unit and the pixels detected in the second area set by the second area setting unit.