We report SARI [1]},
which averages the F1/precision of n-grams \((n \in \lbrace 1, 2, 3, 4\rbrace )\)  inserted, deleted and kept when compared to human references. More specifically, it computes the F1 score for the n-grams that are added (add),We slightly improved the SARI implementation by [1]} to exclude the spurious ngrams while calculating the F1 score for add. For example, if the input contains the phrase “is very beautiful”, the phrase “is beautiful” is treated as a new phrase in the original implementation even though it is caused by the delete operation. which is an important indicator if a model is good at paraphrasing. The model's deletion capability is measured by the F1 score for n-grams that are kept (keep) and precision for those deleted (del).SARI score of a reference with itself may not always be 100 as it considers 0 divided by 0 as 0, instead of 1, when calculating n-gram precision and recall. This avoids the inflation of del scores when the input is same as the output. To evaluate a model's paraphrasing capability and diversity, we calculate the BLEU score with respect to the input (s-BL), the percentage of new words (%new) added, and the percentage of system outputs identical to the input (%eq). Low s-BL, %eq, or high %new indicate that the system is less conservative.
We also report Flesch-Kincaid (FK) grade level readability [3]}, average sentence length (SLen), the percentage of splits (%split), compression ratio (CR), and average output length (OLen). We do not report BLEU because it often does not correlate with simplicity [4]}, [5]}, [1]}.
<TABLE>