It is challenging to learn effective representations from the long time series as modeling latent patterns from high temporal complexity is hard. Recurrent neural networks (RNNs) have been widely used for learning dynamics from the sequential input. However, hundreds of time steps prohibit RNNs from learning accurate representation, due to the vanishing gradient issue. A common approach to tackle the long input sequence for RNNs is to add convolutional layers before the recurrent layers [1]}, [2]}. However, the introduction of a stack of convolutional layers before recurrent layers increases the complexity leading to vanishing gradients.
Another alternative to RNN is the attention approach. Attention, e.g., Transformer [3]}, can capture salient data patterns by skipping recurrent connections, thus avoiding the vanishing gradient issue brought by the long-term dependencies. Nevertheless, pure attention models cannot exploit long-term progression patterns of intraoperative time series, which are informative given the nature of physiological changes during the operation.
