One of the primary objectives of a distributed storage system is to reliably store large amounts of source data for long durations using a large number @math of unreliable storage nodes, each with @math bits of storage capacity. Storage nodes fail randomly over time and are replaced with nodes of equal capacity initialized to zeroes, and thus bits are erased at some rate @math . To maintain recoverability of the source data, a repairer continually reads data over a network from nodes at a rate @math , and generates and writes data to nodes based on the read data. The distributed storage source data capacity is the maximum amount of source data that can be reliably stored for long periods of time. We prove the distributed storage source data capacity asymptotically approaches @math as @math and @math grow. This equation expresses a fundamental trade-off between network traffic and storage overhead to reliably store source data.
Each fragment is partitioned into sub-fragments for the @cite_10 , and the number of sub-fragments provably grows quickly as @math grows or as the @math approaches zero. For example there are @math sub-fragments per fragment for @math and @math , and the generates a fragment at a new node from receiving @math non-consecutive sub-fragments over an interface from each of @math nodes. Accessing many non-consecutive sub-fragments directly from a node is sometimes efficient, in which case the amount of data read over interfaces from nodes is equal to the amount of accessed data used to create the read data, but typically it is more efficient to access an entire fragment at a node and locally select the appropriate sub-fragments to send over the interface from the node to the repairer, in which case the amount of data locally accessed at each node is @math times the size of the data counted as read over the interface from the node.
@cite_10
2492240243
This paper presents an explicit construction for an @math regenerating code over a field @math operating at the Minimum Storage Regeneration (MSR) point. The MSR code can be constructed to have rate @math as close to @math as desired, sub-packetization given by @math , for @math , field size no larger than @math and where all code symbols can be repaired with the same minimum data download. The construction modifies a prior construction by Sasidharan et. al. which required far larger field-size. A building block appearing in the construction is a scalar MDS code of block length @math . The code has a simple layered structure with coupling across layers, that allows both node repair and data recovery to be carried out by making multiple calls to a decoder for the scalar MDS code. While this work was carried out independently, there is considerable overlap with a prior construction by Ye and Barg. It is shown here that essentially the same architecture can be employed to construct MSR codes using vector binary MDS codes as building blocks in place of scalar MDS codes. The advantage here is that computations can now be carried out over a field of smaller size potentially even over the binary field as we demonstrate in an example. Further, we show how the construction can be extended to handle the case of @math under a mild restriction on the choice of helper nodes.