Finding duplicates.ExactSubstr concatenates all the documents in the dataset to create a single long text sequence; then, it builds a suffix array (Manber and Myers, 1993) in linear time--an array of the indexes to a lexicographical ordering of all the suffixes in the sequence. Finally, duplicate sequences can also be found in linear time using the suffix array, by simply traversing the ordered list of suffixes and comparing the beginning of each pair of two consecutive suffixes.