Vaswani et al. [1]} introduced an attention mechanism to model dependencies in sequences regardless of the distances between positions of the input elements. Often such attention mechanisms are combined with RNNs [1]}. Temporal pattern attention LSTM (TPA-LSTM) [3]} utilizes an attention mechanism that allows the model not only to attend to relevant previous times but also to identify interdependencies among multiple features. The dual-stage attention-based recurrent neural network (DA-RNN) [4]} incorporates a dual-stage attention scheme consisting of an encoder with an input attention mechanism and a decoder with a temporal attention mechanism.
