We propose two scalable techniques to detect and remove duplicated training data. _Exact_ substring matching identifies verbatim strings that are repeated. This allows us to identify cases where only part of a training example is duplicated (SS4.1). _Approximate_ full document matching uses hash-based techniques (Broder, 1997) to identify pairs of documents with high \(n\)-gram overlap (SS4.2).