TECHNICAL FIELD

Embodiments of the invention are directed to a system for creating an indexing framework of an input file, such as a document, or of an input string, and, more particularly, to a system or computer implemented method for identifying data element(s)/substructure(s) in a data structure, particularly section(s) and/or numbered list(s) in a document or input document string, and/or for extracting named entities.

BACKGROUND

A named entity is a textual content that identifies an entity by name. For example, named entities may include persons (such as a person's given name or role), organizations (such as the name of a corporation, institution, association, government or private organization), places (locations), temporal expressions, such as specific dates, and monetary expressions.

Named entities are of great interest for the task of information extraction in general, and for many other text processing applications. Identifying a textual content as a named entity can provide additional information about the sentence and the document in which it is being used. Techniques for recognizing named entities in text typically rely on a lexicon which indexes entries that are named entities as such, and may further apply grammar rules, such as requiring capitalization, or use statistical analysis, to confirm that the group of words should be tagged as a named entity.

Automated systems for recognizing named entities are able to extract named entities from digital documents and classify those named entity mentions into one or more pre-specified categories such as person, city, dates, and others. Named entity results may then be used for many downstream purposes such as improving information retrieval systems, knowledge extraction systems and many others.

Certain automated systems can extract tagged (or labeled) information on the named entities based on the extraction of semantic relations, in particular for fact extraction, question-answering, information retrieval, semantic network constructions, ontology building, and the like. Associating such tags (or labels) with the extracted information is an important part of the extraction of named entities. For example, in a textual extract “the commencement date of the present Agreement is Jan. 1, 2020”, a label “Commencement Date” may be associated to a named entity “Jan. 1, 2020”. An essential requirement of the information tags is that they must correctly reflect the meaning of the named entities in the documents and/or of the relationship between the named entities and the documents. A desirable feature of the information tags is that they should match with standard information that, for example, a user might expect to find in certain documents that correspond to certain standard document types or models (for example, a technical specification of a display is expected to contain the display size). A further desirable feature of the information tags is that they should be as much concise as possible to increase the efficiency of comprehension of the information by the user, and the efficiency of storing, indexing, and searching the information in computer systems.

There is an ongoing need to improve the accuracy of existing automated systems for tagging named entities. At present, the usage of many existing named entity tagging systems is restricted to specific or niche type of documents. Also, many existing named entity tagging systems operate in English but not in other languages. There is a need to scale up named entity recognition systems to operate in many different human languages and with many different document types. Moreover, the scaling process requires a large amount of training data, which is usually created manually and hence becomes a costly and time-consuming task. Also, the training processes of many existing named entity tagging systems are extremely computationally expensive.

Embodiments described herein address these and other limitations of the prior art.

BRIEF SUMMARY OF THE INVENTION

The following presents a simplified summary of various aspects described herein. This summary is not an extensive overview, and is not intended to identify key or critical elements or to delineate the scope of the claims. The following summary merely presents some concepts in a simplified form as an introductory prelude to the more detailed description provided below.

Aspects described herein relate to extracting named entities from one or more documents. One or more document strings corresponding to the one or more documents may be generated. A plurality of enumerable markers of the one or more document strings may be identified. Identifying the plurality of enumerable markers may be based on a set of marker identification criteria. A plurality of candidate named entities of the one or more document strings may be identified. Identifying the plurality of candidate named entities may be based on a set of named entity identification criteria. At least one candidate named entity of the plurality of candidate named entities may be associated with at least one enumerable marker. Associating a given candidate named entity with a respective enumerable marker of a document string of the one or more document strings may be based on a proximity of the at least one given candidate named entity to the respective enumerable marker in the document string. A plurality of marker proximity strings may be identified based on a set of proximity string identification criteria. Each marker proximity string of the plurality of marker proximity strings may be associated with at least one associated enumerable marker. The marker proximity string may be a first substring of a document string of the one or more document strings. The first substring may comprise at least a second substring adjacent to the at least one associated enumerable marker. A plurality of named entity descriptions corresponding to a plurality of meanings of the plurality of candidate named entities may be determined. Determining each named entity description of the plurality of named entity descriptions may be based on at least one marker proximity string. Each named entity description may be associated with at least one respective candidate named entity. The at least one respective candidate named entity may be associated with at least one associated enumerable marker. The at least one associated enumerable marker may be associated with the at least one marker proximity string. The determined named entity descriptions and the determined candidate named entities associated with the named entity descriptions may be output.

Methods, including computer-implemented methods, apparatuses, systems, and computer-readable media are also in the scope of the present disclosure.

