(ii) Time series require longer sequences to learn the temporal patterns.
In natural languages, sequences of hundreds of lengths have contained rich semantic information.
Thus, language pre-training models usually cut or pad the input sequence to hundreds [1]}, [2]}, [3]}.
However, although time series have relatively more straightforward semantics than natural languages, they require longer sequences to learn it. For example, traffic system records data every five seconds, and if we want to learn the weekly periodicity, we need at least consecutive 2016 time slices.
Although sampling at a lower frequency is a possible solution, it inevitably loses information.
Fortunately, although longer time series will increase the model complexity, we can alleviate it by stacking fewer Transformer blocks and fix model parameters during the forecasting stage to reduce computational and memory overhead.
