1. An image matching device comprising: a three-dimensional model data memory unit for storing identification information of each of a plurality of parts forming an aggregate and information of positions of said parts in a three-dimensional space; a parts data memory unit for storing data of said parts; an image data memory unit for storing image data of an image taken of said aggregate, information of a position and a direction at which and in which said image was taken, the stored image data and the stored information of the position and the direction being associated one another and stored; image synthesizing means for producing a synthesized image as viewed from a position and in a direction same as those of an image stored in said image data memory unit by referring to said three-dimensional model data memory unit and said parts data memory unit; and means for matching image data of the produced synthesized image to image data of an image stored in said image data memory unit.