1. A method for cataloging a data set using a computer processor, the method comprising the steps of: (a) accessing a plurality of documents stored in the data set; (b) processing the data and text and content of each of the plurality of documents; (c) creating a document profile for each of the plurality of documents processed, using relevant document metrics; (d) assigning weighting coefficients to each of the relevant document metrics based upon a plurality of data set attributes; (e) assigning a unique identifier to each of the plurality of documents processed; (f) mapping the document profile data using standardization algorithms; (g) removing duplicate documents based upon the mapped document profiles; (h) integrating the document profile data for consistency using the plurality of data set attributes; (i) creating a document profile database using the plurality of data set attributes; (j) indexing said document profile database; and (k) updating said document profile database periodically to incorporate new data set data.