[1]} is created based on method adopted by autoencoder to train a vision transformer. Distinctly, the input image is first masked by up to \(75\%\) . These masking strategies applied on images include random, block-wise, and grid. Then, the masked images are encoded into features by a ViT and the decoder decodes the features by another ViT into the original non-mask images. Astonishingly, this strategy ends up with an unprecedentedly high accuracy and the masked images can be well reconstructed.
