1. A pattern inspection method comprising: imaging a first pattern formed in a first area of a substrate to acquire a first pattern image; associating the first pattern image with design information for the first pattern to detect a defect by a die-to-database inspection; comparing the number of the detected defect with a prepared first threshold, and when the number of the detected defects is equal to or less than the first threshold, outputting the detected defect as an inspection result; imaging a second pattern to acquire a second pattern image when the number of the detected defect is more than the first threshold, the second pattern being formed in a second area different from the first area within the substrate and being designed to comprise the same shape as the first pattern; and comparing the first pattern image with the second pattern image to further detect a defect by a die-to-die inspection, and outputting the further detected defects as an inspection result.