TECHNICAL FIELD

The disclosure herein relates to distributed computing, particularly relates to persisting data structures on an asymmetric non-volatile memory storage architecture.

BACKGROUND

Modern computing systems typically involve using one or more computer servers to serve many clients. Today, more and more computer servers are hosted by data centers, which have many computers and associated data storages. With the emergence of non-volatile memory (NVM) devices, conventional hard drives, disks or tapes used in data centers are being replaced with NVM based storages (e.g., NVM based Solid State Drives (SSDs)). Most existing computing systems have a symmetric architecture, in which one NVM based storage (e.g., SSD) is attached to one server machine so that CPU of the server machine may treat the NVM like part of the local memory and store and read the NVM using load and store instructions just like accessing local memory. In the symmetric architecture, the NVM and its data will be unavailable if the server machine goes down. Therefore, there is a need for an asymmetric architecture in which one or more NVM based storages may become a shared resource among servers and still be available to other servers even one or more server fails.

SUMMARY

Disclosed herein is a method, comprising, at a front-end node in a computing system, generating a data structure operation record for a data structure operation, appending the data structure operation record in an operation log, generating a transaction record for a transaction, appending the transaction record in a transaction log, flushing the operation log to a back-end node of the computing system, and flushing the transaction log to the back-end node after flushing the operation log. The data structure operation may be directed to a data structure persisted in a non-volatile memory (NVM) in the back-end node and the data structure operation record may include an operation type and parameter(s) for the data structure operation. The transaction may include a plurality of memory operations that collectively accomplish the data structure operation on the data structure persisted in the NVM and the transaction record may include a plurality of memory operation records corresponding to the plurality of memory operations. The method may further comprise, at the back-end node of the computing system, persisting the operation log received from the front-end node in the NVM, persisting the transaction log received from the front-end node in the NVM and performing memory updates according to the plurality of the memory operation records to accomplish the data structure operation, with the data structure operation record in the operation log received by the back-end node serving as a commit signal for the transaction.

Disclosed herein is a computing system that may comprise a front-end node that comprises at least one central processing unit (CPU), a dynamic random-access memory (DRAM) and a first Remote Direct Memory Access (RDMA) enabled Network Interface Card (RNIC). The at least one CPU at the front-end node may be configured to: generate a data structure operation record for a data structure operation, append the data structure operation record in an operation log, generate a transaction record for a transaction, append the transaction record in a transaction log, flush the operation log to the back-end node via the first RNIC and flush the transaction log to the back-end node via the first RNIC after flushing the operation log. The data structure operation may be directed to a data structure persisted in the NVM and at least partially cached in the DRAM. The data structure operation record may include an operation type and parameter(s) for the data structure operation. The transaction may include a plurality of memory operations that collectively accomplishing the data structure operation on the data structure persisted in the NVM, the transaction record including a plurality of memory operation records corresponding to the plurality of memory operations. The computing system may further comprise a back-end node that comprises a second RNIC and a non-volatile memory (NVM). The second RNIC at the back-end node may be configured to: persist the operation log received from the front-end node in the NVM, persist the transaction log received from the front-end node in the NVM and perform memory updates according to the plurality of the memory operation records to accomplish the data structure operation, with the data structure operation record in the operation log received by the back-end node serving as a commit signal for the transaction.

Disclosed herein is one or more computer-readable non-transitory media comprising one or more instructions that when executed by one or more processors is to configure the one or more processors to perform operations comprising: at a front-end node in a computing system: generating a data structure operation record for a data structure operation, the data structure operation being directed to a data structure persisted in a non-volatile memory (NVM) in a back-end node of the computing system, the data structure operation record including an operation type and parameter(s) for the data structure operation; appending the data structure operation record in an operation log; generating a transaction record for a transaction, the transaction including a plurality of memory operations that collectively accomplishing the data structure operation on the data structure persisted in the NVM, the transaction record including a plurality of memory operation records corresponding to the plurality of memory operations; appending the transaction record in a transaction log; flushing the operation log to the back-end node; and flushing the transaction log to the back-end node after flushing the operation log; and at the back-end node of the computing system: persisting the operation log received from the front-end node in the NVM; persisting the transaction log received from the front-end node in the NVM; and performing memory updates according to the plurality of the memory operation records to accomplish the data structure operation, with the data structure operation record in the operation log received by the back-end node serving as a commit signal for the transaction.

