1. A method of inspecting a specimen, comprising: illuminating a specimen, on which plural patterns are formed, with light emitted from a light source; capturing a first image of a first area of the specimen in which patterns are formed; capturing a second image of a second area of the specimen in which patterns which are essentially the same with the patterns formed in the first area, are formed; creating data relating to pixels of the first image and corresponding pixels of the second image, for each pixel; determining a threshold for each pixel for detecting defects directly in accordance with the first image and the second image; and detecting defects on the specimen by processing said first image and the second image by using said threshold for each pixel and information of a scattered diagram of brightness of the first image and the second image, wherein the threshold is determined by using information of brightness of a local region of at least one of the first image and the second image, with the local region including an aimed pixel and peripheral pixels of the aimed pixel.