To address these challenges, we propose a novel framework, in which STGNN is Enhanced by a scalable time series Pre-training model (STEP).
The pre-training model aims to efficiently learn the temporal patterns from very long-term historical time series and generate segment-level representations, which contain rich contextual information that is beneficial to address the first challenge.
In addition, the learned representations of these segments (i.e., the short-term time series) are able to incorporate the information from the whole long historical time series to calculate the correlation between time series, thus solving the second challenge, the problem of missing the dependency graph.
Specifically, we design an efficient unsupervised pre-training model for Time Series based on TransFormer blocks [1]} (TSFormer), which is trained through the masked autoencoding strategy [2]}.
TSFormer efficiently captures information over very long-term historical data over weeks, and produces segment-level representations that correctly reflect complex patterns in time series.
Second, we design a graph structure learner based on the representation of TSFormer, which learns discrete dependency graph and utilizes the \(k\) NN graph computed based on the representation of TSFormer as a regularization to guide the joint training of graph structure and STGNN.
Notably, STEP is a general framework that can extend to almost arbitrary STGNNs.
In summary, the main contributions are the following:
