1. A method for filtering tokens from a first document for determining whether the first document describes substantially similar subject matter to the subject matter described by a second document, wherein each of the first and second documents is organized into a plurality of fields, and wherein at least some of the fields of both the first document and the second document include tokens representing subject matter described by their respective document, the method comprising: obtaining the first document; selecting a field of the first document and selecting a token from the selected field, such that the selected token has the highest inverse document frequency (IDF) within the selected field with regard to a document corpus; temporarily removing those tokens from the first document that have a higher IDF than the selected token; generating a score based in part upon at least one remaining token that is common to both the first and second documents; determining whether the first document describes substantially similar subject matter to the subject matter described by the second document, wherein the determining comprises evaluating the score with respect to a threshold value; and providing an indication as to whether the first document describes substantially similar subject matter to the subject matter described by the second document according to the determination.