TECHNICAL FIELD

The subject application generally relates to storing data in data storage systems, and more particularly to a preliminary protection technology that uses multiple composite copies of data for improved node failure toleration, and related embodiments.

BACKGROUND

Contemporary cloud-based data storage systems, such as ECS (formerly known as ELASTIC CLOUD STORAGE) provided by DELL EMC, can be based on a cluster of nodes that each owns some part of the stored data (and can store redundant data and/or erasure coded data for data protection purposes) in storage devices. In ECS, disk space is partitioned into a set of blocks referred to as chunks, which for example can be a fixed size of 128 MB. User data objects are stored in chunks, and a data object of typical size ordinarily comprises a set of segments stored to different chunks. The chunks are shared, in that a chunk typically contains segments of multiple, different user objects.

Chunk content is modified in append-only mode. When a chunk becomes full enough, the chunk gets sealed and can no longer be written to with further data. The content of a sealed chunk is immutable.

ECS is a reliable storage, including that erasure coding is used to protect user data at the chunk level. However, chunks are filled with user data at different rates, whereby in general it is difficult to predict the moment when a given chunk will get sealed. During data writes for a client, the data storage system does not send any acknowledgement to the client until the data is properly protected in a non-volatile memory. Therefore, there is a time window between the moment the user data comes into the system and the moment that the chunk gets sealed so that the chunk's content can be encoded.

During this time window, triple mirroring can be used as a preliminary protection scheme before erasure coding occurs; in other words, delayed erasure coding is implemented. Note that with triple mirroring, three mirror copies of a chunk are stored to three different nodes. Therefore, with the basic triple mirroring the system can tolerate dual-node failure.

Triple mirroring results in significant internal data traffic. To reduce the internal data traffic, an optimized triple mirroring preliminary protection scheme was implemented, in which there two complete chunk copies are stored to different nodes, with a third copy comprising a composite copy stored as a set of k data fragments distributed among the nodes; (where k+m refers to erasure coding with k data fragments plus m coding fragments). However, if the two nodes that contain the two complete chunk copies fail, (and at least one of also contained one of the data fragments), the complete set of data is lost. Thus, the data is more vulnerable with this optimized preliminary protection scheme, as this optimized scheme cannot tolerate dual-node failure.

