Computational Approaches to Measuring the Similarity of Short Contexts : A Review of Applications and Methods <s> Micro View of Context <s> This paper describes an experimental comparison of three unsupervised learning algorithms that distinguish the sense of an ambiguous word in untagged text. The methods described in this paper, McQuitty's similarity analysis, Ward's minimum-variance method, and the EM algorithm, assign each instance of an ambiguous word to a known sense definition based solely on the values of automatically identifiable features in text. These methods and feature sets are found to be more successful in disambiguating nouns rather than adjectives or verbs. Overall, the most accurate of these procedures is McQuitty's similarity analysis in combination with a high dimensional feature set. <s> BIB001 </s> Computational Approaches to Measuring the Similarity of Short Contexts : A Review of Applications and Methods <s> Micro View of Context <s> This paper systematically compares unsupervised word sense discrimination techniques that cluster instances of a target word that occur in raw text using both vector and similarity spaces. The context of each instance is represented as a vector in a high dimensional feature space. Discrimination is achieved by clustering these context vectors directly in vector space and also by finding pairwise similarities among the vectors and then clustering in similarity space. We employ two different representations of the context in which a target word occurs. First order context vectors represent the context of each instance of a target word as a vector of features that occur in that context. Second order context vectors are an indirect representation of the context based on the average of vectors that represent the words that occur in the context. We evaluate the discriminated clusters by carrying out experiments using sense–tagged instances of 24 SENSEVAL2 words and the well known Line, Hard and Serve sense–tagged corpora. <s> BIB002 </s> Computational Approaches to Measuring the Similarity of Short Contexts : A Review of Applications and Methods <s> Micro View of Context <s> Determining the similarity of short text snippets, such as search queries, works poorly with traditional document similarity measures (e.g., cosine), since there are often few, if any, terms in common between two short text snippets. We address this problem by introducing a novel method for measuring the similarity between short text snippets (even those without any overlapping terms) by leveraging web search results to provide greater context for the short texts. In this paper, we define such a similarity kernel function, mathematically analyze some of its properties, and provide examples of its efficacy. We also show the use of this kernel function in a large-scale system for suggesting related queries to search engine users. <s> BIB003
The micro view of the context in which a word occurs is represented by a vector that indicates the words that co-occur in relatively close proximity, often within 5 or 10 positions of each other, although the actual size of this window can vary considerably. The values in the micro context vector can take many forms. They may be frequency counts that show how often the two words occur together, or binary values that indicate whether they do or not. These can also be measures of association between the word and each of its cooccurrences, indicating how strongly associated they are, in order to separate those words that occur together just by chance from those that have some systematic relationship. The micro view might be based on the co-occurrence behavior of the word in one very large corpus where it occurs many times, or collected across hundreds of short contexts where it only occurs once. The micro-vector only shows the co-occurring words, it does not indicate where those co-occurrences may have been observed. The co-occurrence counts or association values that make up a word vector based on micro context can be obtained locally from the contexts that are being measured for similarity, or globally from some external corpora that is independent of the shorts contexts to be represented. For example, suppose there are 10 headed short contexts that contain the target word cricket and are to be clustered based on their similarity (perhaps to determine if they pertain to the insect or the sport of the same name). In all likelihood the amount of cooccurrence data that can be obtained from just 10 short contexts is very small, so a global approach that draws co-occurrence information from larger external corpora would be advisable. In this case one possibility would be that cooccurrence information for all the words that appear in the short contexts of cricket could be obtained globally from a large sample of newspaper text. If however there are 10,000 short contexts, it might be possible to obtain sufficient co-occurrence data (in the form of counts or association scores) from the data itself, thus suggesting a local approach. Some tasks will inherently have only a small number of contexts available, such as those that compare a context to a reference sample. In those cases a global method for obtaining cooccurrence data is almost always required. In addition to drawing upon corpora for co-occurrence data for micro context vectors, the Web also presents an abundant and readily available global source of co-occurrence data. Each word from a collection of short contexts could be used as a search query to the Web either in whole or to some specific part of it such as Wikipedia, Google News, etc. The pages returned from that query would then be used to derive the co-occurrence data for that word (e.g., BIB003 Regardless of where the co-occurrence information comes from, a word vector based on micro context will show the company a word keeps, and uses this as a unique representation of that word. Words that have very similar co-occurrences are judged to be similar since they keep the same company, and perhaps could be substituted for each other (a common test of synonymy). Note that there is some granulation in how similar such words are, and that depends on the size of the window used to determine co-occurrence. If any word that occurs within 50 positions of a word is considered a co-occurrence, then quite a few words may show signs of similarity, perhaps at a topic level. However, if a co-occurrence must occur within two positions of the word, then the similarity is more likely to approach synonymy. The micro view of context has been widely used in the Word Sense Disambiguation literature (c.f.,  ). Both BIB001 and BIB002 have shown the micro contexts provide sufficient information for good results in automatic word sense discrimination.