It was shown in previous works such as [1]} that self-supervised learning usually benefits from longer training compared to fully supervised learning. In Table REF , we show the impact of different number of epochs for pre-training our model. We find that the accuracy improves significantly when the number of epochs increases from 100 to 500. However, only slight improvements are achieved when training it for another 500 epochs (for a total of 1000 epochs). So, for our final model, we pre-train the model for 1000 epochs.
