Table REF  shows the impact of different mask strategies. We train DeiT-S with random mask strategy[1]}, attention-guided mask strategy and no mask. For fair comparison, all methods mask with the same probability \(p\) . It can be observed that the performance of random mask strategy degrades. This strategy would probably suppress the ability to recognize the object in the images (from 73.1 to 63.2). Random mask strategy may destroy the tokens of crucial regions of original image which may be indispensable for recognizing object. The masked input may have incomplete or even misleading information. On the contrary, the performance of our attention-guided mask strategy has a steady improvement (from 73.1 to 73.7). Essential regions are mostly preserved, which could be a strong proof of our hypothesis.
