Training for Quantized ViT. 

Knowledge distillation is an essential supervision approach for training quantized neural networks, which bridges the performance gap between quantized models and full-precision counterparts. The usual practice is using distillation through attention as described in [1]}
\(\begin{aligned}\mathcal {L}_{\operatorname{dist}} &= \frac{1}{2} \mathcal {L}_{\operatorname{CE}}(\psi (Z_q), y) + \frac{1}{2} \mathcal {L}_{\operatorname{CE}}(\psi (Z_q), y_t), \\y_t &= \mathop {\arg \max }_c Z_t(c).\end{aligned}\) 
