Computational Approaches to Measuring the Similarity of Short Contexts : A Review of Applications and Methods <s> Macro View of Context <s> How do people know as much as they do with as little information as they get? The problem takes many forms; learning vocabulary from text is an especially dramatic and convenient case for research. A new general theory of acquired similarity and knowledge representation, latent semantic analysis (LSA), is presented and used to successfully simulate such learning and several other psycholinguistic phenomena. By inducing global knowledge indirectly from local co-occurrence data in a large body of representative text, LSA acquired knowledge about the full vocabulary of English at a comparable rate to schoolchildren. LSA uses no prior linguistic or perceptual similarity knowledge; it is based solely on a general mathematical learning method that achieves powerful inductive effects by extracting the right number of dimensions (e.g., 300) to represent objects and contexts. Relations to other theories, phenomena, and problems are sketched. <s> BIB001 </s> Computational Approaches to Measuring the Similarity of Short Contexts : A Review of Applications and Methods <s> Macro View of Context <s> The goal of the on-going project described in this paper is evaluation of the utility of Latent Semantic Analysis (LSA) for unsupervised word sense discrimination. The hypothesis is that LSA can be used to compute context vectors for ambiguous words that can be clustered together --- with each cluster corresponding to a different sense of the word. In this paper we report first experimental result on tightness, separation and purity of sense-based clusters as a function of vector space dimensionality and using different distance metrics. <s> BIB002
The macro view of context adjusts the maxim about judging a word based on the company it keeps. Instead, it holds that : "you can judge a word by the places where it is seen." For example, the word polymer will tend to be restricted to texts about molecular structure or science, whereas the word be will appear in nearly any written text. Most words are of course somewhere in the middle of this spectrum. Macro contexts can be obtained locally from the contexts that are being clustered, or from some other global external source. If there are 10,000 short contexts to be clustered, then it may be that simply recording in which of these contexts each word occurs will provide sufficient information to make a discrimination decision. However, if the task is to discriminate among 10 instances of cricket or to compare a single context to a few reference samples, then a global collection of contexts external to the problem should be used. Possible sources of macro contexts could include textbooks or encyclopedias, where each paragraph is treated as a unit of context in which a word may be seen. The macro view of context has been used extensively in Latent Semantic Analysis BIB001 , where it has been applied to problems in automated essay grading, synonym identification, and short passage similarity measurement, among many others. BIB002 uses the macro view of context for word sense discrimination, and reports encouraging results. The distinction between the micro and macro view of context is subtle, in that words that occur in specific types of macro contexts will likely occur in specific micro contexts as well. However, a word with multiple senses might have very different friends depending on the places it is seen, that is to say the word might have different circles of friends when it is used in different senses. When interest is used in contexts about business it might well refer to money paid for the use of money, and be surrounded by words like rate and mortgage, whereas when used in more general contexts it could be surrounded by anything a person is interested in, such as art, music, sports, etc. Micro context captures the localized behavior of a word, while the macro context in which a word occurs may make a more general topic level distinction. A word can be represented by a vector that shows the words it cooccurs with (micro), or a vector that shows the contexts in which it occurs (macro). In either case, these word vectors are used to create a representation of the short contexts that is more amenable to measuring similarity between short contexts than first-order methods.