1. A method for inspection guided overlay metrology, comprising: performing a pattern search in order to identify one or more instances of a predetermined pattern on a semiconductor wafer; generating a care area for each of the one or more instances of the predetermined pattern on the semiconductor wafer; identifying one or more defects within each of the one or more generated care areas by performing an inspection scan of each of the one or more generated care areas, wherein the inspection scan includes a low-threshold inspection scan or a high sensitivity inspection scan; identifying one or more overlay sites of at least some of the one or more instances of the predetermined pattern of the semiconductor wafer having a measured overlay error larger than a selected overlay specification utilizing a defect inspection technique; comparing location data of the one or more identified defects of a generated care area to location data of the one or more identified overlay sites within the generated care area in order to identify one or more locations wherein the one or more defects are proximate to the one or more identified overlay sites; and generating a metrology sampling plan based on the identified one or more locations wherein the one or more defects are proximate to the one or more identified overlay sites.