1. A defect observation method in a defect observation apparatus having an image obtaining unit which obtains an image of a sample and a signal processing unit which includes a storage unit and a computing unit, receives an input regarding image information from the image obtaining unit, and observes a defect on the sample with a process by the computing unit, the method comprising the steps of: receiving an input regarding design information of the sample to be observed from the storage unit; receiving, based on the design information including one or more layers, an input regarding observable layer information previously set to the sample to be observed from the storage unit; receiving an input regarding defect coordinates on the sample detected by another inspection apparatus from the storage unit; analyzing, for a defect on the sample to be observed based on the defect coordinates, a circuit pattern structure in a peripheral area of the defect coordinates based on the design information and the layer information; estimating a non-defective product image based on a result of the circuit pattern structure analysis; and detecting a defect by a comparative inspection between the non-defective product image and the image information from the image obtaining unit.