For VQ-VAE training, all image data are scaled to \(64 \times 64\)  with patch size \(=8\) , resulting in a total visual token length \(64/8 \times 64/8 \times 5 = 320\)  per image sequence. The number of visual tokens is set to be 256.
For the transformer, we use a 6-layer transformer model with dimension size of 768.
We set the head number as 6 and train a sparse transformer [1]} with a dropout probability of \(0.1\) .
Exact architectural details and hyperparameters can be found in Appendix REF .
