1. A template matching apparatus, containing at least a processor, for matching an input image obtained by photographing an inspection object with a template image, comprising: template area selecting means for selecting a template area from an image for a template as a template original image; similar image detecting means for detecting a similar image having a high correlation to the template original image selected by said template area selecting means, from said image for a template; difference image forming means for forming a difference image between said template original image and said similar image; modified template forming means for modifying said template original image so as to incur a penalty to said similar image during evaluation and forming a modified template, in accordance with said difference image; and similarity degree evaluating means for calculating said modified template and a partial image in each area of said input image and evaluating a similarity degree of each area, in accordance with a predetermined evaluation formula.