The proposed approach to leverage the noisy pseudo-labels relies on findings from [1]}, which show that, while neural-networks are in principle capable of memorizing noisy samples, they tend to first take advantage of shared patterns across training examples, given their finite capacity. In a parallel study, [2]} empirically confirmed, in the classification task, that neural-networks can generalize well even when trained on massively noisy data, rather than just memorizing noise, assuming that the noise on a pseudo-label is not conditioned by the corresponding input image itself. We define this condition as the unpredictability property.
