There are several estimators of conditional probability from observed frequencies of features. In this paper, we propose using the lower limit of confidence interval on posterior distribution determined by the observed frequencies to ascertain conditional probability. In our experiments, this method outperformed other popular estimators.
Church and Hanks @cite_8 , proposed the concept of mutual information to measure the association between words. This method solves low-frequency problem, because infrequent features cannot sum up to a large mutual information quantity. However, if the true strength of the relationship between two features can be regarded as conditional probability, it may not be an appropriate measure to use because mutual information is symmetric towards two features, whereas conditional probability is asymmetric.
@cite_8
1593045043
The term word association is used in a very particular sense in the psycholinguistic literature. (Generally speaking, subjects respond quicker than normal to the word nurse if it follows a highly associated word such as doctor. ) We will extend the term to provide the basis for a statistical description of a variety of interesting linguistic phenomena, ranging from semantic relations of the doctor nurse type (content word content word) to lexico-syntactic co-occurrence constraints between verbs and prepositions (content word function word). This paper will propose an objective measure based on the information theoretic notion of mutual information, for estimating word association norms from computer readable corpora. (The standard method of obtaining word association norms, testing a few thousand subjects on a few hundred words, is both costly and unreliable.) The proposed measure, the association ratio, estimates word association norms directly from computer readable corpora, making it possible to estimate norms for tens of thousands of words.