1. A defect observation method in a defect observation apparatus having an image obtaining unit which obtains an image of a sample and a signal processing unit which includes a storage unit and a computing unit, receives an input regarding image information from the image obtaining unit, and observes a defect on the sample with a process by the computing unit, the method comprising the steps of: receiving an input regarding design information of the sample to be observed from the storage unit; receiving, based on the design information including one or more layers, an input regarding observable layer information previously set to the sample to be observed from the storage unit; receiving an input regarding defect coordinates on the sample detected by another inspection apparatus from the storage unit; analyzing, for a defect on the sample to be observed based on the defect coordinates, a circuit pattern structure in a peripheral area of the defect coordinates based on the design information and the layer information; taking an image so as to include the defect based on the defect coordinates; estimating a non-defective product image based on a result of the circuit pattern structure analysis and the image including the defect; and detecting a defect by a comparative inspection between the non-defective product image and the image including the defect, wherein in the circuit pattern structure analysis, the peripheral area centered on the defect coordinates is divided into localized local areas, the respective local areas are searched for those designed to have a similar appearance, and information of a size and coordinates of the local area obtained as a result of the searching is stored as an analysis result in the storage unit.