A Survey of Text Mining Techniques and Applications <s> V. WEB MINING: KNOWLEDGE DISCOVERY FROM WEB <s> The network structure of a hyperlinked environment can be a rich source of information about the content of the environment, provided we have effective means for understanding it. We develop a set of algorithmic tools for extracting information from the link structures of such environments, and report on experiments that demonstrate their effectiveness in a variety of context on the World Wide Web. The central issue we address within our framework is the distillation of broad search topics, through the discovery of “authorative” information sources on such topics. We propose and test an algorithmic formulation of the notion of authority, based on the relationship between a set of relevant authoritative pages and the set of “hub pages” that join them together in the link structure. Our formulation has connections to the eigenvectors of certain matrices associated with the link graph; these connections in turn motivate additional heuristrics for link-based analysis. <s> BIB001 </s> A Survey of Text Mining Techniques and Applications <s> V. WEB MINING: KNOWLEDGE DISCOVERY FROM WEB <s> When using traditional search engines, users have to formulate queries to describe their information need. This paper discusses a different approach to Web searching where the input to the search process is not a set of query terms, but instead is the URL of a page, and the output is a set of related Web pages. A related Web page is one that addresses the same topic as the original page. For example, www.washingtonpost.com is a page related to www.nytimes.com, since both are online newspapers. We describe two algorithms to identify related Web pages. These algorithms use only the connectivity information in the Web (i.e., the links between pages) and not the content of pages or usage information. We have implemented both algorithms and measured their runtime performance. To evaluate the effectiveness of our algorithms, we performed a user study comparing our algorithms with Netscape’s ‘What’s Related’ service ( http://home. netscape.com/escapes/related/). Our study showed that the precision at 10 for our two algorithms are 73% better and 51% better than that of Netscape, despite the fact that Netscape uses both content and usage pattern information in addition to connectivity information. © 1999 Published by Elsevier Science B.V. All rights reserved. <s> BIB002 </s> A Survey of Text Mining Techniques and Applications <s> V. WEB MINING: KNOWLEDGE DISCOVERY FROM WEB <s> The huge amount of unstructured data available on the Web and the intranets creates an information overloading problem. So, managing the knowledge contained in the textual documents is an important problem of Knowledge Management. Knowledge Extraction from collections of data is possible by Knowledge Discovery in Database (KDD), an interactive and iterative process focused on the exploration of data to discover new and interesting patterns within them. The fundamental phase of KDD process is Data Mining if data are in structured form and Text Mining when they are unstructured. This paper describes a prototype of a vertical corporate portal that implements a KDD process for knowledge extraction from unstructured data contained in textual documents. Text mining is realized through a clustering method that produces a partition of a set of documents on the basis of their contents characterized through the frequency of the words. <s> BIB003
Buried in the enormous, heterogeneous and distributed information on the web is knowledge with great potential value. With the rapid development of the web, it is urgent and important to provide users with tools for efficient and effective resource discovery and knowledge discovery on the web. Although the web search engine assists in resource discovery, it is far from satisfying for its poor precision. Moreover, the target of the web search engine is only to discover resource on the web. As far as knowledge discovery is concerned, it is not equal to at all even with high precision. Therefore, the research and development of new technology further than resource discovery is needed. Data mining is used to identify valid, novel, potentially useful and ultimately understandable pattern from data collection in database community. However, there is little work that deals with unstructured and heterogeneous information on the Web. Web mining  ,  , BIB001 is the activity of identifying patterns implied in large document collection. Web mining is an integrated technology in which several research fields are involved, such as data mining, computational linguistics, statistics, informatics and so on. Different researchers from different communities disagree with each other on what web mining is exactly. Since web mining derives from data mining. Its definition is similar to the well-known definition of data mining. Nevertheless, Web mining has many unique characteristics compared with data mining. Firstly, the source of Web mining are web documents BIB002 . Web mining is to explore interesting information and potential patterns from the contents of web page, the information of accessing the web page linkages and resources of e-commerce by using techniques of data mining, which can help people extract knowledge, improve web sites design, and develop ecommerce better. Using Knowledge Discovery in Database (KDD)  , where the fundamental step is Data Mining, knowledge workers can obtain important strategic information for their business. KDD has deeply transformed the methods to interrogate traditional databases, where data are in structured form, by automatically finding new and unknown patterns in huge quantity of data. However, structured data represent only a little part of the overall organization knowledge. Knowledge is incorporated in textual documents. The amount of unstructured information in this form, accessible through the web, the intranets, the news groups etc is enormously increased in last years. In this scenario the development of techniques and instruments of Knowledge Extraction, that are able to manage the knowledge contained in electronic textual documents, is a necessary task. This is possible through a KDD process based on Text Mining. A particular Text Mining approach is based on clustering techniques used to group documents according to their content. Knowledge discovery from texts (KDTs) involves discovering interesting unseen patterns in text databases, establishing which kind of knowledge should be acquired and how its novelty and interestingness should be evaluated is still a matter of research. The metrics commonly used come from data mining (DM) or knowledge discovery in databases (KDDs) techniques and were developed for structured databases. However, they cannot be immediately applied to text data. Despite the large amount of research over the last few years, only few research efforts worldwide have realized the need for high-level representations (i.e., not just keywords), for taking advantage of linguistic knowledge and for the specific purpose of producing and assessing the unseen knowledge. The rest of the effort has concentrated on doing text mining from an information retrieval (IR) perspective and so both representation (keyword based) and data analysis are restricted. The most sophisticated approaches to text mining or KDT are characterized by an intensive use of external electronic resources, which highly restrict the application of the unseen patterns to be discovered VI. CONCLUSIONS At last we conclude that, Text mining is also known as Text Data Mining or Knowledge-Discovery in Text (KDT)  , refers generally to the process of extracting interesting and non-trivial information and knowledge from unstructured text. Text mining is a young interdisciplinary field which draws on information retrieval, data mining, machine learning, statistics and computational linguistics. As most information (over 80%) is stored as text, text mining is believed to have a high commercial potential value. Knowledge may be discovered from many sources of information, yet, unstructured texts remain the largest readily available source of knowledge. The problem of Knowledge Discovery from Text (KDT) is to extract explicit and implicit concepts and semantic relations between concepts using Natural Language Processing (NLP) techniques. Its aim is to get insights into large quantities of text data. KDT, while deeply rooted in NLP, draws on methods from statistics, machine learning, reasoning, information extraction, knowledge management, and others for its discovery process. KDT plays an increasingly significant role in emerging applications, such as Text Understanding. Most of Text Mining operations are as follows: • Feature Extraction. • Text-base navigation. • Search and Retrieval • Categorization (Supervised Classification) • Clustering (Unsupervised Classification) • Summarization Primary objective of the Feature Extraction operation is to identify facts and relations in text. Most of the times this includes distinguishing which noun phrase is a person, place, organization or other distinct object. Feature Extraction algorithms may use dictionaries to identify some terms and linguistic patterns to detect others. The Text-base navigation enables users to move about in a document collection by relating topics and significant terms. It helps to identify key concepts and additionally presents some of the relationships between key concepts. Search and Retrieval is used for searching internal documents collections. Its main characteristic is the various text search options. After indexing which is the first step, a wide range of text search options may be utilized. These include basic search options such as Boolean (and/or/not), wildcard, segment, numeric range, etc. as well as some advance search capabilities, such as relevancy-ranked natural language searching, fuzzy search, concept search etc. Categorization is the operation that we use, when we want to classify documents into predefined categories. Due to this, we are able to identify the main topics of a document collection. The categories are either preconfigured (by the programmer) or left for the user to specify. A Cluster is a group of related documents, and Clustering BIB003 is the operation of grouping documents on the basis of some similarity measure, automatically without having to pre-specify categories. The most common Clustering algorithms that are used are hierarchical, binary relational, and fuzzy. Hierarchical clustering creates a tree with all documents in the root node and a single document in each leaf node. The intervening nodes have several documents and become more and more specialized as they get closer to the leaf nodes. It is very useful when we are exploring a new document collection and want to get an overview of the collection. The most important factor in a Clustering algorithm is the similarity measure. All Clustering algorithms are based on similarity measures. Summarization is the operation that reduces the amount of text in a document while still keeping its key meaning. With this operation the user usually is allowed to define a number of parameters, including the number of sentences to extract or a percentage of the total text to extract. Trend Analysis is used for identifying trends in documents collected over a period of time. Trends can be used, for example to discover that, a company is shifting interests from one domain to another. In Attribute Analysis, given a set of documents, identify relationships between attributes (features that have been extracted from the documents) such as the presence of one pattern implies the presence of another pattern. Visualization utilizes feature extraction and key term indexing in order to build a graphical representation of the document collection. This approach supports the user in identifying quickly the main topics or concepts by their importance on the representation. Additionally, it is easy to discover the location of specific documents in a graphical document representation. The ability to visualize large text data sets lets users quickly explore the semantic relationships that exist in a large collection of documents. In-formation visualization for text mining typically involves producing a 2D or 3D representation that exposes selected types of semantic patterns in a document collection. To visualize text documents, we first need to convey to the analyst the underlying relation-ships between documents in a geometrically intuitive manner-we must preserve certain Visualization characteristics for the rendering to be meaningful. For example, documents that are close to each other in content should also be geometrically close to each other. Additionally, the analyst should be able to provide his or her own insight in determining what it means for documents to have close content or similar meaning. For users to provide their own insight, they have to access the meaning of the Visualization. That is, users must be able to interpret the rendered data so that the topicdocument relations are clearly defined. research. His current areas of research are-Natural Language Processing and Optical Character recognition. He has published more than 25 research papers in various international and national journals and refereed conferences. He has been actively involved in technical development of Punjabi and has to his credit the first Gurmukhi OCR, Punjabi word processor with spell checker and various transliteration software. He was the chief coordinator of the project "Resource Centre for Indian Language Technology Solutions-Punjabi", funded by the Ministry of Information Technology as well as the coordinator of the Special Assistance Programme (SAP-DRS) of the University Grants Commission (UGC), India. He was also awarded a research project by the International Development Research Centre (IDRC) Canada for Shahmukhi to Gurmukhi Transliteration Solution for Networking.