BACKGROUND

This disclosure is directed to employing machine learning models to optimize database management. In particular, techniques are disclosed for utilizing machine learning models for deciding whether a new database record needs to be created (e.g., to avoid duplicates) and what record to create.

SUMMARY

Database systems are critical in all areas of information technology. As the capacity to store and transmit data continues to grow, efficiently managing and storing such data has become an increasingly important and challenging task for businesses and consumers alike. For an organization (e.g., a corporation, a private entity or a governmental entity), managing a large database presents a multitude of problems, such as determining whether incoming information already exists in the database, determining whether incoming information is appropriate for the particular database, and determining what to create as a database record based on incoming information (e.g., creating a database record to most accurately reflect incoming information while complying with requirements for the particular database). The creation of unneeded database entries (e.g., creating a duplicate record of a database record that already exists in the database, or creating a record that does not belong in the database) unnecessarily consumes finite storage space of the database, and consequently may degrade processing times and performance of the system. Worse, a database that includes multiple entries for the same data may output inconsistent responses to the same queries. Further, if a new database record is incorrectly determined to already exist in the database (e.g., incorrectly matched to an existing database record), and thus is not added to the database, important information could be omitted from the database, which is highly undesirable, as this undermines the reliability of the database and user expectations of comprehensive and accurate database records (e.g., for an organization).

In one approach to managing database storage, a person may review incoming database records and manually match new database records to existing records, or decide what portions of a newly received database record are to be included in a database. However, such a process may be expensive, labor-intensive and prone to human error. Another approach utilizes a system that refers to sets of pre-set heuristic rules in order to determine whether a new database record matches an existing record, or what type of database record should be created. However, such rules may be overly rigid, such as being overly influenced by minor differences in syntax between database records, or overly lax, such as failing to detect important differences between database records.

To overcome these problems, systems and methods are provided herein for updating a local database when a new database record is received using machine learning models. Systems and methods described herein also provide a match machine learning model for calculating respective probabilities that a newly received database record individually matches respective candidate database records identified in the local database. The probabilities output by the match machine learning model may be leveraged by an in-database machine learning model to additionally calculate a probability that the received database record is in the local database (e.g., even if it is uncertain which particular record in the local database corresponds to the received record).

In addition, an out-of-policy machine learning model may be used to calculate a probability that the newly received database record complies with inclusion rules of the local database. Moreover, individual match likelihood (e.g., highest individual match likelihood), the probability that the received database record already exists in the local database, and the probability that the received record is out of policy may be used jointly for the system to make an inclusion decision. For example, each of the probabilities may be combined to determine not only whether the received database record already exists in the local database but also whether the received database record is an appropriate database record for the local database.

When a decision is made to include a record into the database, the database system needs to determine what record to actually create. This decision may be particularly difficult if multiple similar records are received from a remote database or databases. For this purpose, systems and methods described herein provide a selector machine learning model for generating a new database record by selectively choosing attributes from multiple received or incoming database records, to enhance the likelihood that the new database record is accurate and conforms to the local database. For example, once it is determined (e.g., by the match machine learning model, the in-database machine learning model, and the out-of-policy machine learning model, respectively) that the received database record should be created as a new database record and is appropriate for the local database, the selector machine learning model generates a new database record having data items selectively chosen from data items of the multiple received database records.

In some aspects of the disclosure, a database management application is provided that uses a match machine learning model to calculate a probability that a received database record matches a candidate database record identified in a local database. The match machine learning model is trained to accept as input a pair of database records (e.g., computed features scores associated with metadata related to media assets) and output a probability of the database records matching. The database management application further utilizes an in-database machine learning model to accept as input a plurality of probabilities (e.g., computed statistical scores based on such probabilities) computed by the match machine learning model and output a probability that a newly received database record is in the local database.

The database record is received (e.g., by the database management application) from a remote database, and candidate database records in the local database are identified as potentially matching the database record. The database management application may generate candidate lists of features scores by, for each identified candidate database record, generating a respective list of features scores based on the respective candidate database record and the received database record, and each feature score may be associated with a label (e.g., a media asset title). The database management application calculates a plurality of match probabilities by, for each respective candidate database record, inputting the respective candidate database record and the newly received database record into the match machine learning model, to calculate a respective probability of the respective candidate database record matching the newly received database record.

A probability that the received database record is in the local database is calculated by inputting in the in-database machine learning model the plurality of match probabilities (e.g., as a list of statistical scores). For example, there may be a circumstance where two or more of the match probabilities of the candidate database are relatively high, but it is uncertain which of such candidate database records is the one that matches the received record. In such circumstance, a level of confidence could nonetheless be derived that a database record (e.g., one of the two records) is the one that matches the received database record exists in the local database, thereby making it unnecessary to create a new record based on the received record.

The database management application may determine, based on the calculated probability, that a new record for the local database should be created, and the new record for the local database is generated based on the received database record. In this way, received data (e.g., data received from a provider of a metadata asset) may be reconciled with data that already exists in the local database. Thus, the risk of creating duplicate database records in the local database may be minimized, thereby saving storage space, potentially reducing memory requirements of the system, and enabling faster search times than if the duplicate database records had been present in the local database.

In some embodiments, an out-of-policy machine learning model is utilized to calculate a probability of the received database record failing to comply with the inclusion policy rules, in addition to employing the aforementioned match machine learning model and the in-database machine learning model. The database management application determines a combined probability including a largest of the match probabilities amongst the respective probabilities, the probability that the received database record is in the local database, and the probability of the received database record failing to comply with the inclusion policy rules. Based on such combined probability the database management application may determine that a new record for the local database should be created, and such new record is generated for, and stored in, the local database based on the received database record. In this way, multiple machine learning models may be utilized in combination to enhance the likelihood that a new database entry created for the local database complies with relevant policies associated with the database and is not a duplicate of a record already existing in the database.

In some aspects of this disclosure, a selector machine learning model is trained to accept as input a list including labels respectively associated with a set of features scores, and output for each respective label an identifier of one score of the set of features scores associated with the respective label. A database management application determines that a single local database record should be created in the local database based on received database records, where each database record comprises data items associated with each of the labels. An aggregated list is generated (e.g., by the database management application) including each of the labels. For each respective label, such aggregated list includes a respective set of features scores computed based on data items associated with the respective label of the database records.

The aggregated list is input into the selector machine learning model to generate for each respective label an identifier of one of the received database records. A new record for the local database is generated by selecting, for each respective data item of the new record, a data item from a database record of the received database records. Each selected database record is identified by the generated identifier and corresponds to the label that is associated with the respective data item. The new record is then stored in the local database. Accordingly, a single database record may be accurately created based on multiple received database entries (e.g., from multiple remote databases associated with providers of media asset metadata) by selecting a desirable attribute for each label of the generated database record.

The match machine learning model may be trained using training example database record pairs, where each training example database pair is associated with an indicator indicating whether the training example database pair constitutes a previously confirmed match (e.g., confirmed by a human reviewer).

The in-database machine learning model may be trained using training example metadata database records, where each training example metadata database record is associated with an indicator indicating whether the training example constitutes a previously confirmed match (e.g., confirmed by a human reviewer).

The out-of-policy machine learning model may be trained using a plurality of training examples, where each training example includes an indicator indicating whether the training example metadata pair fails to comply with the inclusion policy (e.g., previously confirmed by a human reviewer).

The selector machine learning model may be trained using an aggregated list of features scores, where such aggregated list includes, for each respective label, a list of multiple features scores, extracted from a set of training examples.

The combined probabilities (e.g., output by the match machine learning model, the in-database machine learning model, and the out-of-policy machine learning model) may be compared to a threshold error rate. In response to determining that the combined probability is lower than the threshold error rate, the database management application may determine that the new record for the local database should be created.

Lists of features scores may be created based on the newly received database record and each respective candidate record identified in the local database. The newly received database record and each respective candidate record may be input to the match machine learning model as the respective lists of features scores to calculate the respective probabilities of the new record matching the identified candidate database records, to receive the plurality of match probabilities. Based on such probabilities, a list of statistical scores (e.g., mean, maximum, minimum, standard deviation, variance as between the plurality of match probabilities) may be computed. The plurality of match probabilities may be input to the in-database machine learning model as the list of statistical scores to calculate a probability that the received database record already exists in the local database.

In some embodiments, each database record includes metadata for a media asset, where the metadata may include various metadata items related to the media asset, and each label (e.g., title, description, genre, duration, release date, etc.) is associated with respective metadata items (e.g., respective data values for the aforementioned labels) of the metadata items. The features scores of the database records may correspond to a similarity score for each respective label, as between metadata of the respective candidate database record and the received database record. The candidate database records may be identified in the database based on, e.g., a media asset having a similar title to the media asset contained in the received database record.

Prior to the generating of the list of statistical scores based on the plurality of match probabilities, the database management application may determine, based on the calculated plurality of match probabilities, whether two or more of the candidate database records match the received database record. The database management application may generate the list of statistical scores based on the plurality of match probabilities in response to determining that two or more of the candidate database records match the received database record.

In some aspects of the disclosure, a word embedding machine learning model (e.g., Word2Vec) may be employed (e.g., to determine a feature score for a description label as between database records). The word embedding machine learning model may be trained to generate respective semantic word vectors representing each word in a corpus of words, and the trained word embedding machine learning model is employed to generate respective semantic word vectors.

The new record stored in the local database, and generated based on the output from the selector machine learning model, may include a first data item associated with a first label and a second data item associated with a second label, where the first data item is selected from a different database record of the plurality of received database records than the second data item.

The features scores included in the aggregated list input to the selector machine learning model may be generated based on comparing the respective database records to predetermined rules associated with constructing database records for the local database, and/or may be generated based on comparing the plurality of database records to each other.

