Transformers have also been used to predict cases of influenza [1]} using the data of weekly count of flu cases in a particular area. A single week's prediction is made using ten previous week’s data as the input in the first experiment. The Transformer architecture performed better in terms of ‘root mean square error’ (RMSE) compared to ARIMA, LSTMs, and sequence to sequence models with attention. In the second experiment, Transformer architecture was tested with multivariate time-series data by introducing ‘week number’ as a time-indexed feature and including the first and second-order differences of the time-series as two explicit numerical features. However, this did not show significant improvement in the results. In the third experiment, Time-delay embedding (TDE) is introduced and formed by embedding each scalar input \(x_t\)  into a d-dimensional time-delay space as in Eq. REF .
\(\text{TDE}_{d,\tau } x(t) = \left( x_t, x_{t-1},\cdots ,x_{t-(d-1)\tau } \right)\) 
