FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning
Pseudo labeling and consistency regularization approaches based on confidence
thresholding have made great progress in semi-supervised learning (SSL).
However, we argue that existing methods might fail to adopt suitable thresholds
since they either use a pre-defined / fixed threshold or an ad-hoc threshold
adjusting scheme, resulting in inferior performance and slow convergence. We
first analyze a motivating example to achieve some intuitions on the
relationship between the desirable threshold and model's learning status. Based
on the analysis, we hence propose FreeMatch to define and adjust the confidence
threshold in a self-adaptive manner according to the model's learning status.
We further introduce a self-adaptive class fairness regularization penalty that
encourages the model to produce diverse predictions during the early stages of
training. Extensive experimental results indicate the superiority of FreeMatch
especially when the labeled data are extremely rare. FreeMatch achieves 5.78%,
13.59%, and 1.28% error rate reduction over the latest state-of-the-art method
FlexMatch on CIFAR-10 with 1 label per class, STL-10 with 4 labels per class,
and ImageNet with 100 labels per class, respectively.