1. A method of inspecting patterns, comprising: illuminating a specimen, on which plural patterns are formed, with light emitted from a lamp; detecting a first bright field image of a first area of the specimen in which patterns are formed with an image sensor; detecting a second bright field image of a second area of the specimen in which patterns which are essentially the same patterns formed in the first area are formed with the image sensor; adjusting, by using an image processor, a brightness of at least one of the first bright field image and the second bright field image to match a brightness between the first bright field image and the second bright field image; comparing, by using the image processor, the first bright field image and the second bright field image which are adjusted in brightness to match with each other to detect a defect of the pattern, wherein in the step of adjusting the brightness, by using an image processor, the brightness between the first bright field image and the second bright field image is adjusted by performing a gradation conversion of at least one of the brightness between the first bright field image and the second bright field image; and wherein in the step of comparing, by using an image processor, said defect of the pattern is detected by using information of a scattered diagram of brightness of the first bright field image and the second bright field image.