Decoder.
The decoder is also a series of Transformer blocks that reconstruct the latent representations back to a lower semantic level, i.e., numerical information.
The decoder operates on the full set of patches, including the mask tokens.
Unlike MAE [1]}, we no longer add positional embeddings here since all patches already have positional information added in the encoder.
Notably, the decoder is only used during the pre-training stage to perform the sequence reconstruction task, and can be designed independently of the encoder.
We use only a single layer of Transformer block for balancing efficiency and effectiveness.

Finally, we apply Multi-Layer Perceptions (MLPs) to make predictions whose number of output dimensions equals the length of each patch.
Specifically, given the latent representation \(\mathbf {H}^i_j\in \mathbb {R}^d\)  of patch \(j\) , the decoder gives the reconstructed sequence \(\hat{\mathbf {S}}^i_j\in \mathbb {R}^{LC}\) .
