@inproceedings{
zhou2024continuous,
title={Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition},
author={Zi-Hao Zhou and Siyuan Fang and Zi-Jing Zhou and Tong Wei and Yuanyu Wan and Min-Ling Zhang},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=PaqJ71zf1M}
}
Long-tailed semi-supervised learning poses a significant challenge in training models with limited labeled data exhibiting a long-tailed label distribution. Current state-of-the-art LTSSL approaches heavily rely on high-quality pseudo-labels for large-scale unlabeled data. However, these methods often neglect the impact of representations learned by the neural network and struggle with real-world unlabeled data, which typically follows a different distribution than labeled data. This paper introduces a novel probabilistic framework that unifies various recent proposals in long-tail learning. Our framework derives the class-balanced contrastive loss through Gaussian kernel density estimation. We introduce a continuous contrastive learning method, CCL, extending our framework to unlabeled data using *reliable* and *smoothed* pseudo-labels. By progressively estimating the underlying label distribution and optimizing its alignment with model predictions, we tackle the diverse distribution of unlabeled data in real-world scenarios. Extensive experiments across multiple datasets with varying unlabeled data distributions demonstrate that CCL consistently outperforms prior state-of-the-art methods, achieving over 4% improvement on the ImageNet-127 dataset. The supplementary material includes the source code for reproducibility.
Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition
Zi-Hao Zhou Siyuan Fang Zi-Jing Zhou Tong Wei Yuanyu Wan Min-Ling Zhang