1. A wafer inspection system, comprising: an optical subsystem configured to scan a wafer thereby generating image patches in inspection image frames for the wafer; and one or more computer subsystems coupled to the optical subsystem, wherein the one or more computer subsystems are configured for: aligning each of the image patches in each of the inspection image frames to design information for the wafer; detecting defects in the image patches; deriving multiple layer design attributes at locations of the defects from the image patches corresponding to the locations of the defects; building a decision tree with the multiple layer design attributes, wherein the decision tree is used to separate the defects into bins with different yield impacts on a device being formed on the wafer; and binning the defects with the decision tree.