arXiv:2202.12040v4  [cs.LG]  18 Sep 2023Self-Training: A Survey
Massih-Reza Amini1, Vasilii Feofanov2, Loic Pauletto3,
Li` es Ha
Self-Training: A Survey
Semi-supervised algorithms aim to learn prediction functions from a small set
of labeled observations and a large set of unlabeled observations. Because this
framework is relevant in many applications, they have received a lot of
interest in both academia and industry. Among the existing techniques,
self-training methods have undoubtedly attracted greater attention in recent
years. These models are designed to find the decision boundary on low density
regions without making additional assumptions about the data distribution, and
use the unsigned output score of a learned classifier, or its margin, as an
indicator of confidence. The working principle of self-training algorithms is
to learn a classifier iteratively by assigning pseudo-labels to the set of
unlabeled training samples with a margin greater than a certain threshold. The
pseudo-labeled examples are then used to enrich the labeled training data and
to train a new classifier in conjunction with the labeled training set. In this
paper, we present self-training methods for binary and multi-class
classification; as well as their variants and two related approaches, namely
consistency-based approaches and transductive learning. We examine the impact
of significant self-training features on various methods, using different
general and image classification benchmarks, and we discuss our ideas for
future research in self-training. To the best of our knowledge, this is the
first thorough and complete survey on this subject.
Massih-Reza Amini Vasilii Feofanov Loic Pauletto Lies Hadjadj Emilie Devijver Yury Maximov