1. A defect check device using a SEM, comprising: a scanning electron microscope unit with a table that mounts a sample thereon and is movable within a plane, the sample having a plurality of dies formed therein, the dies each having a pattern of a same shape; a storage unit configured to store a SEM image of the sample that is obtained by imaging the sample mounted on the table by means of the scanning electron microscope unit; an image processing unit configured to process the SEM image of the sample stored in the storage unit and detect a defect on the sample; a user interface unit configured to input a condition for the image processing unit to process the SEM image of the sample, and output a result of the processing; and a control unit configured to control the scanning electron microscope unit, the storage unit, the image processing unit, and the user interface unit, wherein the control unit controls the scanning electron microscope unit to sequentially image a predetermined region of each of a plurality of dies formed on the sample and sequentially obtain the SEM image of the predetermined region of each of the plurality of dies, and store the sequentially obtained SEM images of the predetermined region into the storage unit, and wherein the image processing unit creates reference images using an averaging process on the sequentially obtained SEM images of the predetermined region stored in the storage unit, and compares one of the created reference images with the sequentially obtained SEM images of the predetermined region stored in the storage unit to extract a defect, and classifies the extracted defect; wherein said reference images comprise at least an average image for a center portion of the plurality of dies and an average image for a peripheral portion of the plurality of dies.