A Survey of Text Mining Techniques and Applications <s> C. Summarization <s> We introduced a newly developed text summarization system. It supports both Chinese and English, while this paper focuses on Chinese processing. We apply 6 word level features and 3 sentence level features to weigh each word and sentence. We also describe two new techniques, one is for processing the topic sensitive word feature and another is for processing the sentence length feature. Primary subjective evaluation shows that these approaches are effective and efficient, and performance of the system is promising. <s> BIB001 </s> A Survey of Text Mining Techniques and Applications <s> C. Summarization <s> In this paper we first analyze some state of the art methods for text summarization. We discuss what the main disadvantages of these methods are and then propose a new method using fuzzy logic. Comparisons of results show that our method beats most methods which use machine learning as their core. <s> BIB002
Text summarization is immensely helpful for trying to figure out whether or not a lengthy document meets the user's needs and is worth reading for further information. With large texts, text summarization software processes and summarizes the document in the time it would take the user to read the first paragraph. The key to summarization is to reduce the length and detail of a document while retaining its main points and overall meaning. The challenge is that, although computers are able to identify people, places, and time, it is still difficult to teach software to analyze semantics and to interpret meaning. Generally, when humans summarize text, we read the entire selection to develop a full understanding, and then write a summary highlighting its main points. Since computers do not yet have the language capabilities of humans, alternative methods must be considered. One of the strategies most widely used by text summarization tools, sentence extraction, extracts important sentences from an article by statistically weighting the sentences. Further heuristics such as position information are also used for summarization. For example, summarization tools may extract the sentences which follow the key phrase "in conclusion", after which typically lie the main points of the document. Summarization tools may also search for headings and other markers of subtopics in order to identify the key points of a document. Microsoft Word's AutoSummarize function is a simple example of text summarization. Many text summarization tools allow the user to choose the percentage of the total text they want extracted as a summary. Summarization can work with topic tracking tools or categorization tools in order to summarize the documents that are retrieved on a particular topic. If organizations, medical personnel, or other researchers were given hundreds of documents that addressed their topic of interest, then summarization tools could be used to reduce the time spent sorting through the material. Individuals would be able to more quickly assess the relevance of the information to the topic they are interested in. An automatic summarization BIB002 process can be divided into three steps: (1)In the preprocessing step a structured representation of the original text is obtained;  In the processing step an algorithm must transform the text structure into a summary structure; and (3) In the generation step the final summary is obtained from the summary structure. The methods of summarization can be classified, in terms of the level in the linguistic space, in two broad groups: (a) shallow approaches, which are restricted to the syntactic level of representation and try to extract salient parts of the text in a convenient way; and (b) deeper approaches, which assume a semantics level of representation of the original text and involve linguistic processing at some level. In the first approach the aim of the preprocessing step is to reduce the dimensionality of the representation space, and it normally includes: (i) stop-word elimination -common words with no semantics and which do not aggregate relevant information to the task (e.g., "the", "a") are eliminated; (ii) case folding: consists of converting all the characters to the same kind of letter case -either upper case or lower case; (iii) stemming: syntactically-similar words, such as plurals, verbal variations, etc. are considered similar; the purpose of this procedure is to obtain the stem or radix of each word, which emphasize its semantics. A frequently employed text model is the vector model. After the preprocessing step each text element -a sentence in the case of text summarization -is considered as a N-dimensional vector. So it is possible to use some metric in this space to measure similarity between text elements. The most employed metric is the cosine measure, defined as cos q = (<x.y>) / (|x| . |y|) for vectors x and y, where (<,>) indicates the scalar product, and |x| indicates the module of x. Therefore maximum similarity corresponds to cos q = 1, whereas cos q = 0 indicates total discrepancy between the text elements. To implement text summarization based on fuzzy logic, MATLAB is usually used since it is possible to simulate fuzzy logic in this software. Select characteristic of a text such as sentence length, similarity to little, similarity to key word and etc. as the input of fuzzy system. Then, all the rules needed for summarization are entered in the knowledge base of this system. After ward, a value from zero to one is obtained for each sentence in the output based on sentence characteristics and the available rules in the knowledge base. The obtained value in the output determines the degree of the importance of the sentence in the final summary. The Kernel of generating text summary using sentenceselection based text summarization approach BIB001 is shown in figure7. 