BACKGROUND

The present embodiments relate to cognitive process and information extraction. More specifically, the embodiments relate to dictionary management directed at identification of problematic terms in a dictionary, the problematic terms being ambiguous or spurious with respect to a given corpus. Management of the dictionary, including selective pruning of identified problematic terms facilitates subsequent application of the dictionary with respect to information extraction and associated tasks.

SUMMARY

The embodiments include a system, computer program product, and method for dictionary membership management directed at identifying ambiguity in semantic resources.

In one aspect, a system is provided with an artificial intelligence (AI) platform and one or more associated tools embedded therein for dictionary membership management directed at identifying ambiguity in semantic resources. A processing unit is operatively coupled to memory and is in communication with the AI platform and the embedded tools, including a dictionary manager, a context manager, and a director. The dictionary manager functions to apply a dictionary of seed terms to a text corpus and identify matching items in the corpus. The context manager characterizes the linguistic properties of each matching item in the corpus and constructs a context pattern from the corpus of each matching item. The director applies each constructed context pattern to the text corpus, and identifies and quantifies the matching content between the dictionary terms and the content extracted with the context pattern from the corpus. The dictionary manager identifies lexicon items from the dictionary that have anomalous behavior reflected in the quantification. In response to one or more seed words being identified as having anomalous behavior, and the dictionary manager selectively removes the identified one or more seed words from the dictionary.

In another aspect, a computer program device is provided for dictionary membership management directed at identifying ambiguity in semantic resources. The program code is executable by a processing unit to apply a dictionary of seed terms to a text corpus and identify one or more matching items in the corpus. The program code characterizes linguistic properties of each matching item in the corpus and constructs a context pattern from the corpus of each matching item. The program code applies each constructed context pattern to the text corpus and identifies and quantifies the matching content between the dictionary terms and the content extracted with the context pattern from the corpus. Lexicon items from the dictionary that have anomalous behavior reflected in the quantification are identified. In response to identification of one or more seed words having anomalous behavior, the identified one or more seed words are selectively removed from the dictionary.

In yet another aspect, a method is provided for dictionary membership management directed at identifying ambiguity in semantic resources. A dictionary of seed terms is applied to a text corpus, and one or more matching items in the corpus are identified. Linguistic properties for each matching item in the corpus are characterized, and a context pattern of each matching item is constructed from the corpus. Each constructed context pattern is applied to the text corpus and matching content between the dictionary terms and the content extracted with the context pattern is identified and quantified. Lexicon items from the dictionary that have anomalous behavior reflected in the quantification are identified. In response to one or more seed words being identified as having anomalous behavior, the one or more identified seed words are selectively removed from the dictionary.

These and other features and advantages will become apparent from the following detailed description of the presently preferred embodiment(s), taken in conjunction with the accompanying drawings.

