For the GRU experiments, we use the source code of MSP and OE in [1]}. We follow the same pre-processing steps and the base rate of \(\mathcal {D}_\text{out}^\text{test}\)  to \(\mathcal {D}_\text{in}^\text{test}\)  is 1:5 in each scenario.
We implement ENN, DP, and our model based on the same two-layer GRUs. We pre-train the base classifier for five epochs and fine-tune five more epochs for OE and our model using WikiText-2. Except for DP, we pre-train it for ten epochs to ensure the same accuracy as others. We evaluate our model with auxiliary datasets regularization (+OE).
