In order to transform an image \(X \in \mathbb {R}^{C \times H \times W}\)  into a 1D sequence of vectors, an image will be transformed into \(X_p \in \mathbb {R}^{N \times (P^2C)}\)  as illustrated in Fig. REF , where \(P\times P\)  is the resolution of each patch, and the number of patches is \(N = HW/P^2\) . The patches are then projected into patch embeddings by a linear layer. Akin to BERT [1]}, a learnable class token \(x_{cls} \in \mathbb {R}^{1 \times (P^2C)}\)  is added to \(X_p\) . Afterward, the positional embeddings are added to the output \([x_{cls}; X_p]\)  to obtain the positional information.
