1. A line of sight detecting device, comprising: face direction estimation means for estimating a face direction of an object person based on a shot face image of the object person; eye detection means for detecting a part of an eye outline in the face image of the object person; pupil detection means for detecting a pupil in the face image of the object person; and line of sight direction estimation means for estimating a direction of a line of sight of the object person based on a correlation of a pupil position in the eye outline and the face direction with respect to the direction of the line of sight, wherein the line of sight direction estimation means estimates the line of sight direction of the object person based on the correlation of the pupil position, the face direction and a shape of the eye outline with respect to the line of sight direction, and the pupil position, the face direction and the shape of the eye outline of the object person.