Our base generation model is a Transformer encoder-decoder initialized by the BERT checkpoint [1]}, which achieved the best reported performance on text simplification in the recent work [2]}. We enhance this model with an attention-based copy mechanism to encourage lexical paraphrasing, while remaining faithful to the input.
