A Survey of Text Mining Techniques and Applications <s> E. Clustering <s> The huge amount of unstructured data available on the Web and the intranets creates an information overloading problem. So, managing the knowledge contained in the textual documents is an important problem of Knowledge Management. Knowledge Extraction from collections of data is possible by Knowledge Discovery in Database (KDD), an interactive and iterative process focused on the exploration of data to discover new and interesting patterns within them. The fundamental phase of KDD process is Data Mining if data are in structured form and Text Mining when they are unstructured. This paper describes a prototype of a vertical corporate portal that implements a KDD process for knowledge extraction from unstructured data contained in textual documents. Text mining is realized through a clustering method that produces a partition of a set of documents on the basis of their contents characterized through the frequency of the words. <s> BIB001 </s> A Survey of Text Mining Techniques and Applications <s> E. Clustering <s> With rapid development of Internet information, It is quite an important project for data mining that how to classify these large amounts of texts. In this paper, we propose an improved text classify cluster algorithm, while calculating similarity, we synthetically consider the relationship between keywords and eigenvector representation on base of term frequency statistics, thereby it lessens sensitivity of input sequence and frequency, and effectively raises similarity accuracy of small text and simple sentence as well as preciseness and recall rate of text cluster result. <s> BIB002
Clustering BIB001 is a technique used to group similar documents, but it differs from categorization in that documents are clustered on the fly instead of through the use of predefined topics. Another benefit of clustering is that documents can appear in multiple subtopics, thus ensuring that a useful document will not be omitted from search results. A basic clustering algorithm creates a vector of topics for each document and measures the weights of how well the document fits into each cluster. Clustering technology can be useful in the organization of management information systems, which may contain thousands of documents. In K-means clustering algorithm BIB002 , while calculating Similarity between text documents, not only consider eigenvector based on algorithm of term frequency statistics ,but also combine the degree of association between words ,then the relationship between keywords has been taken into consideration ,thereby it lessens sensitivity of input sequence and frequency, to a certain extent, it considered semantic understanding , effectively raises similarity accuracy of small text and simple sentence as well as preciseness and recall rate of The first step in text clustering is to transform documents, which typically are strings of characters into a suitable representation for the clustering task. (1)Remove stop-words: The stop-words are high frequent words that carry no information (i.e. pronouns, prepositions, conjunctions etc.). Remove stop-words can improve clustering results. (2) Stemming: By word stemming it means the process of suffix removal to generate word stems. This is done to group words that have the same conceptual meaning, such as work, worker, worked and working. (3) Filtering: Domain vocabulary V in ontology is used for filtering. By filtering, document is considered with related domain words (term). It can reduce the documents dimensions. A central problem in statistical text clustering is the high dimensionality of the feature space. Standard clustering techniques cannot deal with such a large feature set, since processing is extremely costly in computational terms. We can represent documents with some domain vocabulary in order to solving the high dimensionality problem. In the beginning of word clustering, one word randomly is chosen to form initial cluster. The other words are added to this cluster or new cluster, until all words are belong to m clusters. This method allow one word belong to many clusters and accord with the fact. This method implements word clustering by calculating word relativity and then implements text classification.