Pseudo-Labeling is a simple approach whereby a model incorporates it's own predictions on unlabeled data to obtain additional information during the training [1]}, [2]}, [3]}, [4]}. The main downside of these methods is that they are unable to correct their own mistakes where predictions of the model on unlabeled data are confident but incorrect. In such a case, the erroneous data not only can not contribute to the training, but the error of the models is amplified during the training as well. This effect is aggravated where the domain of the unlabeled data is different from that of labeled data.
Note that pseudo-labeling in [2]} is similar
to entropy regularization [6]}, in the sense that it forces the model to provide higher confidence predictions for unlabeled data. However, it differs because it only forces these criteria
on data which have a low entropy prediction due to the threshold of confidence.
