1. An endoscope insertion direction detecting device comprising: a classification section for performing, on a scene of an endoscope image of a moving image picked up by an endoscope inserted into a body cavity, classification into classes for a plurality of different feature values relating to detection of an endoscope insertion direction in the body cavity; and an insertion direction computing section, provided for each of the classes for the feature values, into which the classification is performed by the classification section, for computing an insertion direction of the endoscope.