1. An image comparison device comprising: a memory; and a processor coupled to the memory, wherein a model image is stored in the memory, the model image including a position of a feature point of a comparison object and a luminance gradient direction at the position of the feature point; the processor configured with a program to perform operations as functional units comprising: a feature amount acquisition unit configured to acquire the position of the feature point and the luminance gradient direction at the position of the feature point from an input image; and an image comparison unit configured to compare the model image and the input image, wherein the image comparison unit performs image comparison of the model image and the input image in which an allowable range in both the position of the feature point and the luminance gradient direction at the position of the feature point provided.