BACKGROUND

1. Field

This disclosure relates generally to performing computer implemented language processing tasks on regulatory content.

2. Description of Related Art

Governments at all levels generate documents setting out requirements and/or conditions that should be followed for compliance with the applicable rules and regulations. For example, Governments implement regulations, permits, plans, court ordered decrees, and bylaws to regulate commercial, industrial, and other activities considered to be in the public's interest. Standards bodies, companies, and other organizations may also generate documents setting out conditions for product and process compliance. These documents may be broadly referred to as “regulatory content”.

Modern enterprises thus operate under an increasing burden of regulation, which has proliferated exponentially in an attempt by regulatory agencies and other governmental bodies to mitigate potential and actual dangers to the public. Documents setting out regulatory content may vary in size, from one page to several hundred pages. As a result, compliance with regulatory content has become increasingly difficult for enterprises. There remains a need for methods and systems that reduce the burden for enterprises in establishing which regulations and conditions in a body of regulatory content are applicable to their operations.

SUMMARY

In accordance with one disclosed aspect there is provided a computer-implemented method for generating regulatory content requirement descriptions. The method involves receiving requirement data including a plurality of requirements extracted from regulatory content, the requirement data including hierarchical information identifying a hierarchical level of each requirement within the plurality of requirements. The method also involves identifying parent requirements within the plurality of requirements based on the existence of one or more child requirements on a hierarchical level immediately below the parent requirement. The method further involves generating requirement pairs, each pair including one of the parent requirements and at least one of the one or more child requirements on the hierarchical level immediately below the parent requirement. The method also involves feeding each of the requirement pairs through a conjunction classifier, the conjunction classifier having been trained to generate a classification output indicative of the requirement pair being one of not a conjunction (NC) between the parent requirement and the child requirement, a single requirement conjunction (CSR) between the parent requirement and the child requirement, or a multiple requirement conjunction (CMR) between the parent requirement and the child requirement. The method also involves generating a set of requirement descriptions based on the final classification generated for each parent requirement.

Generating the requirement pairs may involve generating a single requirement pair for each parent requirement, the single requirement pair including the parent requirement and all of the child requirements on the hierarchical level immediately below the parent requirement.

Generating the requirement pairs may involve generating a plurality of separate requirement pairs for each parent requirement, each separate requirement pair including the parent requirement and one of the one or more child requirements on the hierarchical level immediately below the parent requirement.

The method may involve generating a final classification for each parent requirement based on a combination of the classification outputs for the requirement pairs corresponding to the one or more child requirements on a hierarchical level immediately below the parent requirement.

Generating the final classification for each parent requirement may involve feeding the classification output for each parent requirement through a final classification neural network, the final classification neural network having been trained to generate the final classification based on the combination of the classification outputs for the requirement pairs.

Generating the final classification may involve assigning a final classification to a parent requirement based on the classifications assigned by the conjunction classifier to the requirement pairs associated with the parent requirement on a majority voting basis.

Generating the final classification may involve assigning a CSR classification to the parent requirement when any one of the classification outputs associated with the requirement pairs is assigned a CSR classification, and if none of the classification outputs associated with the requirement pairs is assigned a CSR classification, assigning a CMR classification to the parent requirement when any one of the classification outputs associated with the requirement pairs is assigned a CMR classification, and if none of the classification outputs associated with the requirement pairs is assigned a CSR or CMR classification, assigning a NC classification to the parent requirement.

Generating the set of requirement descriptions may involve, for each parent requirement assigned a NC classification, generating a requirement description that includes text associated only with the parent requirement, for each parent requirement assigned a CSR classification, generating a single requirement description that concatenates text associated with the parent requirement and each of the one or more child requirements at the hierarchical level below the parent requirement, and for each parent requirement assigned a CMR classification, generating a separate requirement description that concatenates text associated with the parent requirement and the text of each of the one or more child requirements at the hierarchical level below the parent requirement.

The method may involve generating a spreadsheet listing the set of requirement descriptions, each requirement description appearing under a requirement description column on a separate row of the spreadsheet, each row further including the associated citation in a citation column.

Generating the spreadsheet listing may further involve for a parent requirement that is assigned a final classification of CSR, including the associated single requirement description on a spreadsheet row associated with the parent requirement, for a parent requirement that is assigned a final classification of CMR including the separate requirement description for each of the one or more child requirements on a spreadsheet row associated with the respective child requirement, and leaving the requirement description column for the spreadsheet row associated with parent requirement empty.

Generating the spreadsheet listing may further involve, generating a label column, the label column including a requirement label (REQ) for each of a parent requirement that is assigned a final classification of CSR a child requirement associated with a parent requirement assigned a final classification of CMR, and a requirement addressed elsewhere (RAE) label for each parent requirement assigned a final classification of CMR.

Receiving the plurality of requirements may involve receiving regulatory content and generating a language embedding output representing the regulatory content, processing the language embedding output to identify citations and associated requirements within the regulatory content, and processing the plurality of citations to determine a hierarchical level for the citation and associated requirement.

The language embedding may be generated using a pre-trained language model, the language model having been fine-tuned using a corpus of unlabeled regulatory content.

The method may further involve, prior to generating regulatory content requirement descriptions, configuring a conjunction classifier neural network to generate the classification output, the conjunction classifier neural network having a plurality of weights and biases set to an initial value, in a training exercise, feeding a training set of requirement pairs through the conjunction classifier, each requirement pair in the training set having a label indicating whether the pair is a NC, CSR, or CMR requirement pair, and based on the classification output by the conjunction classifier neural network for requirement pairs in the training set, optimizing the plurality of weights and biases to successively train the neural network for generation of the classification output.

The method may involve generating a plurality of requirement summarizations, each requirement summarization corresponding to one of the requirement descriptions and summarizing a text content of the requirement description.

Generating the plurality of requirement summarizations may involve feeding each of the requirement descriptions through a summarization generator, the summarization generator being implemented using a summarization generator neural network that has been trained to generate a summarization output based on a text input.

The method may involve fine-tuning the summarization generator neural network using a regulatory content dataset including requirement descriptions and corresponding requirement description summaries.

The method may involve training the summarization generator neural network by identifying requirements in regulatory content, generating training data in which the identified requirements are masked while leaving descriptive text, optional requirements, and recommendations unmasked, training the summarization generator neural network using the training data, fine-tuning the summarization generator neural network using a regulatory content dataset including requirement descriptions and corresponding requirement description summaries.

The corresponding requirement description summaries may be generated by human review of the regulatory content dataset.

The method may involve training the summarization generator neural network by extracting requirements from a plurality of different regulatory content sources to generate a requirement corpus, generating language embeddings for the requirement sentences in the requirement corpus, identifying similar requirement sentences within the requirement corpus that meet a similarity threshold based on their respective language embeddings, for each of the identified similar requirement sentences, generating a control token that is based on attributes of the requirement sentence to generate labeled training samples for training summarization generator neural network.

In accordance with one disclosed aspect there is provided a system for generating regulatory content requirement descriptions. The system includes a parent/child relationship identifier, configured to receive requirement data including a plurality of requirements extracted from regulatory content, the requirement data including hierarchical information identifying a hierarchical level of each requirement within the plurality of requirements. The parent/child relationship identifier is also configured to identify parent requirements within the plurality of requirements based on the existence of one or more child requirements on a hierarchical level immediately below the parent requirement, and to generate requirement pairs, each pair including one of the parent requirements and at least one of the one or more child requirements on the hierarchical level immediately below the parent requirement. The system also includes a conjunction classifier configured to receive each of the requirement pairs, the conjunction classifier having been trained to generate a classification output indicative of the requirement pair being one of not a conjunction (NC) between the parent requirement and the child requirement, a single requirement conjunction (CSR) between the parent requirement and the child requirement, or a multiple requirement conjunction (CMR) between the parent requirement and the child requirement. The system further includes a requirement description generator, configured to generate a set of requirement descriptions based on the classification output generated for each parent requirement.

The parent/child relationship identifier may be configured to generate the requirement pairs by generating a single requirement pair for each parent requirement, the single requirement pair including the parent requirement and all of the child requirements on the hierarchical level immediately below the parent requirement.

The parent/child relationship identifier may be configured to generate the requirement pairs by generating a plurality of separate requirement pairs for each parent requirement, each separate requirement pair including the parent requirement and one of the one or more child requirements on the hierarchical level immediately below the parent requirement.

The requirement description generator may be configured to generate a final classification for each parent requirement based on a combination of the classification outputs for the requirement pairs corresponding to the one or more child requirements on a hierarchical level immediately below the parent requirement.

The requirement description generator may involve a final classification neural network, the final classification neural network having been trained to generate the final classification based on the combination of the classification outputs for the requirement pairs.

The requirement description generator may be configured to generate the final classification by assigning a CSR classification to the parent requirement when any one of the classification outputs associated with the requirement pairs is assigned a CSR classification, and if none of the classification outputs associated with the requirement pairs is assigned a CSR classification, assigning a CMR classification to the parent requirement when any one of the classification outputs associated with the requirement pairs is assigned a CMR classification, and if none of the classification outputs associated with the requirement pairs is assigned a CSR or CMR classification, assigning a NC classification to the parent requirement.

The system may include a summarization generator operably configured to generate a plurality of requirement summarizations, each requirement summarization corresponding to one of the requirement descriptions and summarizing a text content of the requirement description.

The summarization generator may include a summarization generator neural network that has been trained to generate a summarization output based on a text input.

The summarization generator neural network may be trained by identifying requirements in regulatory content, generating training data in which the identified requirements are masked while leaving descriptive text, optional requirements, and recommendations unmasked, training the summarization generator neural network using the training data, fine-tuning the summarization generator neural network using a regulatory content dataset including requirement descriptions and corresponding requirement description summaries.

The summarization generator neural network may be trained by extracting requirements from a plurality of different regulatory content sources to generate a requirement corpus, generating language embeddings for the requirement sentences in the requirement corpus, identifying similar requirement sentences within the requirement corpus that meet a similarity threshold based on their respective language embeddings, for each of the identified similar requirement sentences, generating a control token that is based on attributes of the requirement sentence to generate labeled training samples for training summarization generator neural network.

Other aspects and features will become apparent to those ordinarily skilled in the art upon review of the following description of specific disclosed embodiments in conjunction with the accompanying figures.

