Semi-Supervised Learning (SSL) provides a framework for utilizing large amount of unlabeled data when obtaining labels is expensive [1]}, [2]}, [3]}. SSL techniques have been shown to improve deep models performance across different machine learning tasks including text classification, machine translation, image classification [4]}, [5]}, [6]}, [7]}, [8]}, [9]}. A common practice to evaluate SSL algorithms is to take an existing labeled dataset and only use a small fraction of training data as labeled data, while treating the rest of the data as unlabeled dataset. Such evaluation, often constrained to the cases when labeled data is scarce, raises questions about the usefulness of different SSL algorithms in a real-world setting [10]}.
