We follow<cite> Monroe et al. (2017)</cite> for language model hyper-parameters, with embedding and LSTM layers of size 100. Also following this prior work, we use a learning rate of 0.004, batch size 128, and apply 0.5 dropout to each layer. We train for 7, 000 iterations, evaluating the model's accuracy on the dev set every 100 iterations.