To summarize, data duplication offers significant advantages and no observed disadvantages. In the remainder of this paper we present our text deduplication framework in SS4, and study the extent of duplicate content in common NLP datasets (e.g., C4, Wiki-40B, and LM1B) in SS5. We then examine the impact of deduplication on test perplexity (SS6.1) and on the frequency of emitting memorized content (SS6.2). Finally, we analyze to what extent perplexity on existing, released models are skewed as a result of overlap between the train and test/validation splits (SS6.3).