paragraph40.75ex plus1ex minus.2ex-1emImage Encoder. Given an image \(x\in \mathbb {R}^{3\times H \times W}\) , we flatten the image into \(N = \left\lfloor {\frac{H}{P}}\right\rfloor \cdot \left\lfloor {\frac{W}{P}}\right\rfloor \)  separate patches \(x_i, i \in {1, \dots , N}\)  of size \(P \times P\)  each.
The vision encoder we use is a ViT [1]}, which maps the image patches \(x_i\)  to a vector of \(N\)  spatial tokens \(f(x) = [f(x_{1}), \dots f(x_{N})]\) .
