Self-supervised learning (SlfSL), aiming at learning feature representations through ingeniously designed pretext tasks without human annotation, has achieved compelling progress in the past few years. Very recently, SlfSL has also been identified as a promising solution for semi-supervised learning (SemSL) since it offers a new paradigm to utilize unlabeled data. This work further explores this direction by proposing a new framework to seamlessly couple SlfSL with SemSL. Our insight is that the prediction target in SemSL can be modeled as the latent factor in the predictor for the SlfSL target. Marginalizing over the latent factor naturally derives a new formulation which marries the prediction targets of these two learning processes. By implementing this framework through a simple-but-effective SlfSL approach -- rotation angle prediction, we create a new SemSL approach called Conditional Rotation Angle Prediction (CRAP). Specifically, CRAP is featured by adopting a module which predicts the image rotation angle \textbf{conditioned on the candidate image class}. Through experimental evaluation, we show that CRAP achieves superior performance over the other existing ways of combining SlfSL and SemSL. Moreover, the proposed SemSL framework is highly extendable. By augmenting CRAP with a simple SemSL technique and a modification of the rotation angle prediction task, our method has already achieved the state-of-the-art SemSL performance. The recent success of deep learning is largely attributed to the availability of a large amount of labeled data. However, acquiring high-quality labels can be very expensive and time-consuming. Thus methods that can leverage easily accessible unlabeled data become extremely attractive. Semisupervised learning (SemSL) and self-supervised learning (SlfSL) are two learning paradigms that can effectively utilize massive unlabeled data to bring improvement to predictive models. SemSL assumes that a small portion of training data is provided with annotations and the research question is how to use the unlabeled training data to generate additional supervision signals for building a better predictive model. In the past few years, various SemSL approaches have been developed in the context of deep learning. The current state-of-the-art methods, e.g. MixMatch (Berthelot et al., 2019) , unsupervised data augmentation (Li et al., 2018) , converge to the strategy of combining multiple SemSL techniques, e.g. Π-Model (Laine & Aila, 2017) , Mean Teacher (Tarvainen & Valpola, 2017) , mixup (Zhang et al., 2018) , which have been proved successful in the past literature. SlfSL aims for a more ambitious goal of learning representation without any human annotation. The key assumption in SlfSL is that a properly designed pretext predictive task which can be effortlessly derived from data itself can provide sufficient supervision to train a good feature representation. In the standard setting, the feature learning process is unaware of the downstream tasks, and it is expected that the learned feature can benefit various recognition tasks. SlfSL also offers a new possibility for SemSL since it suggests a new paradigm of using unlabeled data, i.e., use them for feature training. Recent work has shown great potential in this direction. This work further advances this direction by proposing a new framework to seamlessly couple SlfSL with SemSL. The key idea is that the prediction target in SemSL can serve as a latent factor in the course of predicting the pretext target in a SlfSL approach. The connection between the predictive targets of those two learning processes can be established through marginalization over the latent factor, which also implies a new framework of SemSL. The key component in this framework is a module that predicts the pretext target conditioned on the target of SemSL. In this preliminary work, we implement this module by extending the rotation angle prediction method, a recently proposed SlfSL approach for image recognition. Specifically, we make its prediction conditioned on each candidate image class, and we call our method Conditional Rotation Angle Prediction (CRAP). The proposed framework is also highly extendable. It is compatible with many SemSL and SlfSL approaches. To demonstrate this, we further extend CRAP by using a simple SemSL technique and a modification to the rotation prediction task. Through experimental evaluation, we show that the proposed CRAP achieves significantly better performance than the other SlfSL-based SemSL approaches, and the extended CRAP is on par with the state-of-the-art SemSL methods. In summary, the main contributions of this paper are as follows: • We propose a new SemSL framework which seamlessly couples SlfSL and SemSL. It points out a principal way of upgrading a SlfSL method to a SemSL approach. • Implementing this idea with a SlfSL approach, we create a new SemSL approach (CRAP) that can achieve superior performance than other SlfSL-based SemSL methods. • We further extend CRAP with a SemSL technique and an improvement over the SlfSL task. The resulted new method achieves the state-of-the-art performance of SemSL. In this work, we introduce a framework for effectively coupling SemSL with SlfSL. The proposed CRAP method is an implementation of this framework and it shows compelling performance on several benchmark datasets compared to other SlfSL-based SemSL methods. Furthermore, two extensions are incorporated into CRAP to create an improved method which achieves comparable performance to the state-of-the-art SemSL methods.
Coupling semi-supervised learning with self-supervised learning and explicitly modeling the self-supervised task conditioned on the semi-supervised one
{"src": "scitldr-AIC_train", "paper_id": "BJxoz1rKwr"}