1. A pattern inspection method comprising: sequentially imaging plural chips formed on a substrate; selecting a pattern which is suitable for calculating position gap between an inspection image of a subject chip and a reference image stored in a memory from an image of a firstly imaged chip among said sequentially imaged plural chips formed on the substrate; computing position gap between an inspection image of a chip obtained by the sequential imaging and reference image stored in a memory by using a positional information of a pattern image included in the inspection image and a reference pattern image included in the reference image which are both corresponding to the pattern selected at the selecting; aligning the inspection image and the reference image by using information of the calculated position gap; and comparing the aligned inspection image with the reference image and extracting a difference between the two images as a defect candidate.