TECHNICAL FIELD

This disclosure relates generally to replication of data and, more particularly, to an improved method of and system for asynchronous replication of data utilizing an efficient mechanism for tracking status of data replication.

BACKGROUND

The need to replicate data sets is becoming increasingly important. Some common examples include local data replication, server to server data replication, server to client data replication, local to cloud data replication, cloud to cloud data replication and client to client data replication. Local data replication may be used to provide data redundancy in a single system. Server to server replication may provide for greater data redundancy in case of faults, as well as increased data availability, load balancing, and geographic proximity between users and data. Server to client data replication may enable access to replicated data on a client device. For example, replicated data may be accessed on a desktop computer at an office or on a portable laptop computer at a remote location. Client to client replication may provide access to replicated data on multiple client devices. For example, data may be changed on a desktop and replicated on a portable laptop computer at a remote location. Data replication may also be used to replicate data between different types of systems when transferring data from one type of technology for storage to a different type.

When the amount of data that is being replicated is significantly large, for example in large relational data sets with millions of rows per table, replicating the data efficiently can become a challenging task. One of the main sources of difficulty in replicating large data sets is keeping track of the status of data replication. This may become a particularly cumbersome task when a need arises to keep track of any changes to the data as it is being replicated. Because replication can take place over a large span of time and multiple process lifetimes, often times changes may be made to the data as it is being replicated. Keeping track of changes and managing the state of replication at each process lifecycle in an efficient manner may require a substantial amount of resources and could negatively affect performance.

Hence, there is a need for an improved method and system for asynchronous replication of data which uses an efficient mechanism for tracking the status of data replication.

SUMMARY

In one general aspect, the instant application describes a device having a processor and a memory in communication with the processor where the memory comprises executable instructions that, when executed by the processors, cause the device to perform multiple functions. The function may include receiving a list of identifiers for a plurality of entities in a data set in a source database, the list of identifiers identifying the entities in the data set for replication from the source database to a destination database; loading at least a portion of the list into a probabilistic replication filter, the at least one portion including an identifier for each one of the plurality of entities in the data set that have not yet been replicated in the destination database; examining the list of identifiers to determine if an identifier is in the probabilistic replication filter; and upon determining that the identifier is in the probabilistic replication filter, replicating an entity associated with the identifier from the source database to the destination database and removing the identifier from the probabilistic replication filter.

In yet another general aspect, the instant application describes a method for asynchronous replication of data from a source database to a destination database where the method includes the steps of receiving a list of identifiers for a plurality of entities in a data set in the source database; loading at least a portion of the list into a probabilistic replication filter, the at least one portion including an identifier for each one of the plurality of entities in the data set that have not yet been replicated in the destination database; examining the list of identifiers to determine if an identifier is in the probabilistic replication filter; and upon determining that the identifier is in the probabilistic replication filter, replicating an entity associated with the identifier from the source database to the destination database and removing the identifier from the probabilistic replication filter.

In a further general aspect, the instant application describes a non-transitory computer readable medium on which are stored instructions that when executed cause a programmable device to receive a list of identifiers for a plurality of entities in a data set in the source database; load at least a portion of the list into a probabilistic replication filter, the at least one portion including an identifier for each one of the plurality of entities in the data set that have not yet been replicated in the destination database; examine the list of identifiers to determine if an identifier is in the probabilistic replication filter; and upon determining that the identifier is in the probabilistic replication filter, replicate an entity associated with the identifier from the source database to the destination database and removing the identifier from the probabilistic replication filter.

This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure.

