Furthermore, Transformer is a type of self-attention mechanism-based deep neural network that is
wildly used in natural language processing (NLP). A transformer has been developed to satisfy
computer vision because of the strong representation ability recently. The use of Transformer performs
better by comparing with other networks, such as CNN, which has shown strong competitiveness [1]}. A transformer has a more robust ability of global information representation than CNNs, making it possible to describe the microorganism structure in a complete image. Especially, Vision
Transform (ViT) is one of the most remarkable visual transformer method till now, which directly applies
sequences of image patches (with position information) as input first. The ViT projects the patches
to the original transformer encoder and classifies the images with a multi-head attention
mechanism as it does in NLP tasks.
The framework of ViT is shown in Fig. REF .
<FIGURE>