1. An endoscope insertion direction detecting device comprising: a scene feature value computing section for computing a specific feature value on a scene of an endoscope image of a moving image picked up by an endoscope inserted into a body cavity; a classification section for computing a degree of similarity between the scene of the endoscope image and a plurality of training data as references for a plurality of classes, which are respectively defined for different structures in the body cavity, from the specific feature value and the plurality of training data, and classifying the scene of the endoscope image into one or more of the plurality of classes based on the degree of similarity computed; and an insertion direction computing section for computing an insertion direction of the endoscope corresponding to the scene of the endoscope image, with respect to the one or more of the classes into which the scene of the endoscope image is classified, wherein: the classification section performs classification into classes of lumen dark sections, folds, and others as a plurality of specific feature values, and if the scene of the endoscope image is classified into the class of lumen dark sections, the insertion direction computing section computes a region of a lumen dark section based on red image components in the endoscope image, if the scene of the endoscope image is classified into the class of folds, the insertion direction computing section computes a barycentric position of a fold based on the red image components in the endoscope image, and if the scene of the endoscope image is classified into the class of others, the insertion direction computing section computes a density gradient in the endoscope image.