We first experiment on a small subset of our CausalBank corpus (CB_10M) – 10 million CE pairs from the causal pattern `because' – considering different NMT encoder and decoder architectures (LSTM, CNN, Conv-Transformer [1]}, and Transformer).Each of these models' encoder and decoder use the same architecture, e.g. both are 6-layer LSTMs, with a hidden size and embedding size of 512. All models are trained for 10 epochs. The vocabulary size is 10,000. For the cause generation model, \(e\)  is used as the source and \(c\)  is used as the target, which is reversed in training the effect model. Perplexity (Per) and word accuracy (Acc) are used to evaluate the model's performance. We find that Transformer constantly achieves the best performance (Table REF ).
<TABLE>