1. A defect inspection device comprising: illumination unit that irradiates an object to be inspected, with light, the object having patterns formed on a surface; light collecting unit that collects light reflected, diffracted, and scattered from the object irradiated with the light by the illumination unit; optical path branching unit that branches the light collected by the light collecting unit upon receiving the light reflected, diffracted, and scattered from the object into a first detection optical path and a second detection optical path; a first spatial filter fitted with a first light blocking pattern to block specific reflected, diffracted, and scattered light of the reflected, diffracted, and scattered light traveling towards the first detection optical path created as a result of branching by the optical path branching unit; first imaging unit that forms an image from the light passed through the first spatial filter; first image acquisition unit that acquires a first image by detecting the image formed by the first imaging unit; a second spatial filter fitted with a second light blocking pattern different from the first light blocking pattern, to block specific reflected, diffracted, and scattered light of the reflected, diffracted, and scattered light traveling towards the second detection optical path created as a result of branching by the optical path branching unit; second imaging unit that forms an image from the light passed through the second spatial filter; second image acquisition unit that acquires a second image by detecting the image formed by the second imaging unit; and image processing unit that conducts image processing to extract defect candidates by integratedly processing the first image acquired by the first image acquisition unit and the second image acquired by the second image acquisition unit.