We adopt the big transformer configuration following<cite> Vaswani et al. (2017)</cite> , with the dimension of word embeddings, hidden states and non-linear layer set as 1024, 1024 and 4096 respectively.