Fortunately, we can alleviate these problems based on the pre-trained TSFormer.
Motivated by recent works [1]}, [2]}, [3]}, we aim to learn a discrete sparse graph, where \(\mathbf {\Theta }_{ij}\)  parameterizes the Bernoulli distribution from which the discrete dependency graph \(\mathbf {A}\)  is sampled.
First, we introduce graph regularization to provide supervised information for graph optimization based on the representations of TSFormer.
Specifically, we denote \(\mathbf {H}^i=\mathbf {H}^i_1\parallel \mathbf {H}^i_2...\mathbf {H}^i_{P-1}\parallel \mathbf {H}^i_{P}\in \mathbb {R}^{Pd}\)  as the feature of time series \(i\) , where \(\parallel \)  means the concatenation operation.
Then we calculate a \(k\) NN graph \(\mathbf {A}^a\)  among all the nodes. We can control the sparsity of the learned graph by setting different \(k\) .
Benefiting from the ability of TSFormer, \(\mathbf {A}^a\)  can reflect the dependencies between nodes, which is helpful to guide the training of the graph structure.
Then, we compute \(\mathbf {\Theta }_{ij}\)  as follows:
\(\begin{aligned}\mathbf {\Theta }_{ij}&=\text{FC}(\text{relu}(\text{FC}(\mathbf {Z}^i\parallel \mathbf {Z}^j)))\\\mathbf {Z}^i&= \text{relu}(\text{FC}(\mathbf {H}^i)) + \mathbf {G}^i,\end{aligned}\) 
