We follow the commonly used rationalization architectures [1]}, [2]} in our implementations:
We use bidirectional gated recurrent units (GRU) [3]} in the generators and the predictors for both our A2r and our reimplemented Rnp. For A2r, we share the parameters of both predictors' GRU while leaving the output layers' parameters separated.
Our rationale predictor \(\mathbf {f}_r\) encodes the masked input \(\mathbf {M} \odot \mathbf {X}\) into the hidden states, followed by max-pooling.
The attention-based predictor \(\mathbf {f}_a\) encodes the entire input \(\mathbf {X}\) into hidden states, which is then weighted by \(\mathbf {\alpha }\).
