The encoder network is responsible for processing longitudinal measurements of patients and passing the encoded sequence to the decoder network. As mentioned earlier, longitudinal EHRs are very sparse. To handle the missing values of longitudinal measurements, we use GRU-D cells in the first layer of the encoder network. There is often a strong correlation between the missing pattern of covariates and labels in medical datasets. A GRU-D cell learns the relationship between labels and the missing pattern of covariates during a supervised learning process and utilizes the observed relationship to impute missing longitudinal measurements for continuous covariates. GRU-D imputes missing values of a covariate by applying a decay rate to the last measured value of that covariate. The influence of the last covariate measurement is reduced proportionally over time if the covariate has not been measured. If a covariate is not measured for a long period of time, GRU-D imputes that missing value by relying more on the mean value of that covariate rather than its last measurement. The imputation process in GRU-D needs three input vectors: a vector containing longitudinal measurements, a mask vector which indicates if a measurement is available or missing (value one represents a covariate is measured and value zero represents the covariate is missing) and a Delta vector which represents the time difference between the current timestamp and the last timestamp at which a covariate for a patient was measured. Readers can refer to [1]} for the implementation details of GRU-D cells. The encoder network of Survival Seq2Seq can be stacked with multiple RNN layers if desired. In that case, the stacked layers can be made up of some other recurrent cells such as Gated Recurrent Unit (GRU) or Long Short-Term Memory (LSTM). A stacked encoder allows the model to learn more complex temporal relationships in data, provided that the training set is large enough to support a deeper network. If not, the encoder can also work with only one GRU-D layer.
