To demonstrate that our model can be controlled to generate diverse simplifications, we evaluate under the following settings: (i) Standard evaluation on the Newsela-Auto test set similar to the methodology in the recent literature [1]}, [2]}, [3]}, and (ii) Evaluation on different subsets of the Newsela-Auto test set that concentrate on a specific operation. We selected 9,356 sentence pairs with sentence splits for split-focused evaluation. Similarly, we chose 9,511 sentence pairs with compression ratio \(<\)  0.7 and without sentences splits to evaluate delete-focused simplification. We created a new dataset, called Newsela-Turk, to evaluate lexical paraphrasing.We also provide results on 8,371 sentence pairs of Newsela-Auto test set with compression ratio \(>\)  0.9 and no splits in Appendix , which show similar trends.
Similar to the Wikipedia-Turk benchmark corpus [4]}, Newsela-Turk consists of human-written references focused on lexical paraphrasing.
We first selected sentence pairs from the Newsela-Auto test set of roughly similar length (compression ratio between 0.8 and 1.2) and no sentence splits because they more likely involve paraphrasing. Then, we asked Amazon Mechanical Turk workers to simplify the complex sentence without any loss in meaning.We provide instructions in Appendix  To ensure the quality of simplifications, we manually selected the workers using the qualification test proposed in [5]}, during which the workers were asked to simplify three sentences. We selected top 35% of the 300 workers that participated in the test. We periodically checked the submissions and removed the bad workers. In the end, we collected 500 sentences with 4 references for each sentence.
<TABLE><TABLE><TABLE>