BACKGROUND

The present invention generally relates to programmable computing systems, and more specifically, to detecting and processing sections spanning processed document partitions.

Information retrieval computing systems can receive search queries from a user and provide answers back to the user. In information retrieval, a question answering (QA) system is tasked with automatically answering a question posed in natural language to the system. A QA system can retrieve an answer by searching a data corpus for documents matching the search query. To assist the QA system, the documents are annotated to describe relationships between co-existing entities. The process of annotating documents is performed by subject matter experts who review the documents and determine whether any entities in the documents can be annotated. The annotations assist the QA system to retrieve the best documents to answer the search query.

SUMMARY

Embodiments of the present invention are directed to detecting and processing sections spanning processed document partitions. A non-limiting example computer-implemented method includes caching a document partition, the document partition including metadata indicating that the document partition is a portion of a whole document. The method also includes pairing a candidate paragraph from the document partition with a cached paragraph segment and determining, using a coherence model, a probability that the candidate paragraph and the cached paragraph segment constitute a semantically coherent paragraph. The method further includes discarding the cached paragraph segment and processing the candidate paragraph and the cached paragraph segment separately based on a determination that the probability is less than a threshold level and processing the candidate paragraph and the cached paragraph segment together as a cross-partition paragraph based on a determination that the probability is greater than the threshold level.

Other embodiments of the present invention implement features of the above-described method in computer systems and computer program products.

Additional technical features and benefits are realized through the techniques of the present invention. Embodiments and aspects of the invention are described in detail herein and are considered a part of the claimed subject matter. For a better understanding, refer to the detailed description and to the drawings.

