1. A pattern inspection device, comprising: an image picking-up portion for picking-up an image of a pattern formation member in which a plurality of opening patterns are formed so as to obtain a picked-up image of the pattern formation member; a reference image obtaining portion for obtaining a reference image used for comparing with the picked-up image; and a pattern defect detecting portion for matching the center locations of the opening pattern images respectively between the picked-up image and the reference image, forming difference images of the opening pattern images between the picked-up image and the reference image per the opening pattern and detecting the defect of the opening pattern base on the difference images.