1. A visual inspection method for performing a visual inspection of semiconductor devices using a computer system provided with an image detector unit, an image processor unit and a display unit, the visual inspection method comprising: a classification condition setting step of setting classification conditions beforehand; a defect candidate detection step of detecting defect candidates by using inspection images acquired by imaging a target substrate, via the image detector unit, and calculating features of the defect candidates; and a defect candidate classification step of classifying the defect candidates in accordance with the classification conditions set beforehand in the classification condition setting step using the features of the defect candidates calculated in the defect candidate detection step, via the imaging processor unit, wherein said classification condition setting step further comprises: a collection step of detecting a large number of defect candidates by using images acquired by imaging a sample substrate, which are different from the inspection images acquired at the defect candidate detection step, and collecting calculated features of each defect candidates over the large number of defect candidates, and collecting calculated features of each defect candidate over the large number of defect candidates to store collected features of each defect candidates; a defect sampling step of creating a defect feature distribution indicating a defect occurrence distribution based on the collected features of each defect over the large number of defects in the collection step, and performing sampling of review defects based on the defect feature distribution; and a review step of providing at least whether a defect candidate is a defect or not to a plurality of review defects by reviewing the review defects sampled in the defect sampling step, wherein a result of classification of the review defects provided in the reviewing step of features of the review defects collected in the collection step, are set as training data of the classification condition.