We then use ExactSubstr, leveraging the implementation from Lee et al. (2022)++, to identify ranges of exact duplicate text of at least 50 tokens. We experiment with three different approaches for these ranges: ExactSubstr-Cut, where we remove them from the original text, as done in the original implementation; ExactSubstr-Mask, where the dataset is unchanged but we do not compute the loss on the duplicated ranges; and ExactSubstr-Drop, where we simply drop an entire document if the duplicated ranges make up more than a certain percentage of its content.