1. A method for classifying a defect in a defect image of a portion of a scanned substrate from a defect imaging tool, the method comprising: a) generating defect information from the defect image; b) retrieving design information relating to one or more structures to be formed on the substrate in a vicinity of the defect; and c) classifying the defect based on a combination of information from the defect image and design information relating to one or more structures to be formed on the substrate in the vicinity of the defect by aligning the defect image to the design information, generating one or more image masks, calculating histograms for each image mask generated, creating fill layers from the histograms, compositing design information and the fill layers, and generating a synthetic reference image from the design information and the fill layers.