Text compression can be applied to various tasks such as Summarization, Text Editing and even Data Augmentation where the compressed text can be employed as additional training examples. However, almost all existing approaches require either parallel data, hand-crafted rules, or extra syntactic information such as dependency labels or part-of-speech tag features trees (McDonald, 2006; Filippova and Strube, 2008;<cite> Zhao et al., 2018)</cite> .