For the first criterion, we have used the 'WRN-28-2' model (i.e., ResNet with depth 28 and width 2) [1]}, including batch normalization [2]} and leaky ReLU nonlinearities [3]}. We conducted our experiments on the widely used CIFAR-10 [4]}, and SVHN [5]} datasets. Note that in our experiments, we tackle the general SSL challenge where the labeled and unlabeled data come from the same underlying distribution, and a given unlabeled data belongs to one of the classes in the labeled set and therefor, there is no class distribution mismatch. Moreover, for each of these datasets, we split the training set into two different sets of labeled and unlabeled data. For training, we use the well-known Adam optimizer [6]} with the default hyperparameters values and a learning rate of \(3\times 10^{-3}\)  in our experiments, and all the experiments have been done on a NVIDIA TITAN X GPU. The batch size in our experiments is set to 100. We have not used any form of early stopping; however, we have consistently monitored the performance of the validation set and reported test error at the point of lowest validation error. The stopping criteria for the Sinkhorn algorithm is either maxIter \(=\)  10,000 or tolerance \(=10^{-8}\) , where maxIter is the maximum number of iterations and tolerance is a threshold for the integrated stopping criterion based on the marginal differences. In experiments, we followed the data augmentation and standard data normalization used in [7]}. Specifically, for SVHN, we
converted pixel intensity values of the images to floating point values in the range of [-1, 1]. For the data augmentation, we only applied random translation by up to 2 pixels. We used the standard training and validation split, with 65,932 images for the training set and 7,325 for the validation set.
For CIFAR-10, we applied global contrast normalization. The data augmentation on
CIFAR-10 are random translation by up to 2 pixels, random horizontal flipping, and Gaussian
input noise with standard deviation 0.15. We used the standard training and validation split, with 45,000
images for the training set and 5,000 images for the validation set.
