Afterward, they further extracted candidates for complex word by combining word embeddings with WordNet and parallel corpora<cite> [Paetzold and Specia, 2017a]</cite> . After examining existing LS methods ranging from rulesbased to embedding-based, the major challenge is that they generated simplification candidates for the complex word regardless of the context of the complex word, which will inevitably produce a large number of spurious candidates that can confuse the systems employed in the subsequent steps. In this paper, we will first present a BERT-based LS approach that requires only a sufficiently large corpus of regular text without any manual efforts.