In summary, TSFormer is efficient thanks to the high masking ratio and fewer Transformer blocks.
TSFormer is capable of learning from the very long-term sequence (e.g., weeks) and can be trained on a single GPU.
The encoder generates representations for the input patches (segments).
Furthermore, another noteworthy difference from MAE [1]} is that we pay more attention to the representations of the patches.
On the one hand, we can use the representations to verify periodic patterns in the data, which will be demonstrated in Section REF .
More importantly, they can conveniently act as contextual information for short-term input of downstream STGNNs, which will be introduced in the next.
