1. A visual inspection device which compares a captured image of an inspection object with a group of images of items regarded as non-defective items to make a non-defective/defective determination, the device comprising: an image inputting unit for accepting inputs of a plurality of images constituting a group of images of items regarded as non-defective items and storing these images; a threshold setting unit for setting a defect threshold for detecting a defective portion of an inspection object based on the plurality of stored images; a defect amount calculating unit for calculating a defect amount to be compared with a determination threshold for making the non-defective/defective determination on the inspection object with respect to each of the plurality of stored images based on the set defective threshold; an outlier testing unit for testing, by statistical processing whether or not each of the calculated defect amounts is an outlier; and an information displaying unit for displaying and outputting outlier information for specifying an image whose defect amount has been tested to be the outlier.