1. A method of reviewing a defect, comprising: a step of previously performing an inspection for defects on a sample having a pattern formed thereon by different inspection apparatus and, if there is a defect, obtaining position information of the defect, then imaging an inspection region including the defect by use of the position information under a scanning electron microscope (SEM) thereby to acquire a defect image, a step of acquiring a reference image that does not include the defect, the reference image being corresponding to the inspection region, a step of extracting a defective region by comparing the defect image to the reference image, a step of extracting a line edge on the reference image existing in a region corresponding to the extracted defective region, a step of extracting a line edge on the defect image existing in a region corresponding to the line edge, and a step of determining a vertical relationship between the defect and the pattern on the sample from the line edge on the reference image and the line edge on the defect image.