We train and evaluate our models on Newsela [1]}https://newsela.com/data/ and Wikipedia copora [2]}, [3]}, [4]}. Newsela consists of 1,882 news articles with each article rewritten by professional editors for students in different grades. We used the complex-simple sentence pairs automatically aligned by [5]}, called the Newsela-auto dataset.
To capture sentence splitting, we joined the adjacent sentences in the simple article that are aligned to the same sentence in the complex article. Following [6]}, we removed the sentence pairs with high (\(>\) 0.9) and low (\(<\) 0.1) BLEU [7]} scores, which mostly correspond to the near identical and semantically divergent sentence pairs respectively. The final dataset consists of 259,778 train, 32,689 validation and 33,391 test complex-simple sentence pairs, where \(\sim \) 30% of pairs involve sentence splitting. Besides Newsela, we also provide the details of experiments on Wikipedia corpus in Appendix , which show similar trends.
