By progressively changing \(\rho \)  using a PSS, we can train substantially faster than a baseline ViT while achieving similar performance. Figure REF (left) compares the wall clock training time for baseline ViT-B to ViT-B + PSS on ImageNet [1]}. The addition of the PSS leads to a training time reduction of 31% (from 24 to 17 hours) while achieving within 0.3% of the classification accuracy. Figure REF (right) illustrates that the ViT model trained with a PSS is also more robust to using a subset of patches during inference. This improvement is due to the model learning from a varying number of patches across training iterations. For example, when \(\rho \)  = 0.6 (i.e., keeping 60% of the patches), the ViT-B + PSS model achieves an image throughput improvement of 35% compared to \(\rho \)  = 1.0 for a 2.3% accuracy reduction. The main contributions of our work are:
