(4) We try to standardize pre-training data to ImageNet-1k, however BEiT uses the DALL\(\cdot \) E  [1]} discrete VAE (dVAE), which was trained on \(\scriptstyle \sim \) 250 million proprietary and undisclosed images, as an image tokenizer. The impact of this additional training data is not fully understood.
