In this section, we study the impact of the pre-training data on the performance of denoising autoencoder, and how they compare to those of joint embedding methods.
More precisely, we investigate how the number of images, and their nature, influence the quality of self-supervised models.
In this preliminary analysis, we consider the recent method BEiT and SplitMask, our variant as detailed in Section , as representatives of denoising autoencoders, and DINO [1]} of a joint embedding method, respectively.
<TABLE>