1. A pattern inspection apparatus for inspecting a pattern to-be-inspected by using an image of said pattern to-be-inspected and data for fabricating said pattern to-be-inspected, said pattern inspection apparatus comprising: a reference pattern generating device configured to generate a reference pattern represented by one or more line segments, one or more curves, or one or more line segments and one or more curves, from said data; an image generating device configured to generate said image of said pattern to-be-inspected; a detecting device configured to detect an edge of said image of said pattern to-be-inspected; and an inspection device configured to inspect said pattern to-be-inspected by comparing said detected edge of said image of said pattern to-be-inspected with said line segment or said curve of said reference pattern, wherein said inspection device assumes correspondence of edges, which are obtained by converting said reference pattern, to said detected edge of said image of said pattern to-be-inspected, wherein said correspondence-assumption is conducted by considering a distance between each of said edges, which are obtained by converting said reference pattern, and each of said detected edge of said image of said pattern to-be-inspected and a directional difference between each of said edges, which are obtained by converting said reference pattern, and each of said detected edge of said image of said pattern to-be-inspected.