1. A defect classification method to classify defects by using a classifier having a binary tree structure on the basis of features of the defects extracted on the basis of detected signals acquired from a defect inspection apparatus, the defect classification method comprising: a classifier construction process for constructing the classifier having the binary tree structure by setting a branch condition for each branch point in the binary tree structure on the basis of instruction of defect classes and feature data respectively associated therewith beforehand, the branch condition including defect classes respectively belonging to groups located on both sides of the branch point, a feature to be used for branch, and a discriminant reference.