1. A method for determining the likelihood of two documents describing similar subject matter, the method comprising: obtaining a set of tokens for each of a first and a second document, wherein each token in the set of tokens represents a series of characters found in its corresponding document; obtaining a set of matched token pairs, each matched token pair comprising a first token from the set of tokens corresponding to the first document and a second token matching the first token, the second token from the set of tokens corresponding to the second document, each matched token pair having a token similarity score; determining a similarity score for the first and second documents according to the token similarity scores of the token pairs in the set of matched token pairs by dividing the sum of the similarity scores of the token pairs in the set of matched token pairs by the sum of the number of token pairs in the set of matched token pairs, the number of tokens from the first document less the number of token pairs in the set of matched token pairs, and the number of tokens from the second document less the number of token pairs in the set of matched token pairs; and providing the determined similarity score as the likelihood of the first and second documents describing similar subject matter.