1. A SEM-aided defect-reviewing apparatus comprising: defect information storing means that stores position information on defects detected on a sample during inspection with an external inspection apparatus; a scanning electron microscope (SEM) that acquires an image of the sample by irradiating a surface of the sample with an electron beam that has converged and scanning over the surface of the sample; image processing means that processes the image acquired by the SEM; and control means that controls the SEM and the image processing means, wherein: the control means controls the SEM to acquire a low-magnification image of a region including the defects detected on the sample by using the defect position information stored by the defect information storing means; the image processing means compares the low-magnification image acquired by imaging the region inclusive of the defects on the sample by the SEM at the low-magnification, with a reference image of a low magnification to extract the defects, and acquires position information on the defects; the control means further controls the SEM for imaging the region including the defects on the sample with a high-magnification to acquire a high-magnification image of the region including the defects by using the defect position information acquired by the image processing means; and the image processing means further obtains a high-magnification reference image for comparing with the high-magnification image of the region including the defects, by imaging a region included in the low-magnification image including the defects at a high-magnification with the SEM in case searching a region suitable for acquiring the high-magnification reference image from the low-magnification image is succeeded or by shifting the imaging region of the SEM on the sample to a reference image region and imaging the reference image region at a high-magnification with the SEM in case searching the region suitable for acquiring the high-magnification reference image from the low-magnification image is not succeeded, and acquires high-magnification defect images by comparing the obtained high-magnification reference image with the high-magnification image of the region including the defects.