It is a common narrative that blockchains are immutable and so it is technically impossible to erase data stored on them. For legal and ethical reasons, however, individuals and organizations might be compelled to erase locally stored data, be it encoded on a blockchain or not. The common assumption for blockchain networks like Bitcoin is that forcing nodes to erase data contained on the blockchain is equal to permanently restricting them from participating in the system in a full-node role. Challenging this belief, in this paper, we propose and demonstrate a pragmatic approach towards functionality-preserving local erasure (FPLE). FPLE enables full nodes to erase infringing or undesirable data while continuing to store and validate most of the blockchain. We describe a general FPLE approach for UTXO-based (i.e., Bitcoin-like) cryptocurrencies and present a lightweight proof-of-concept tool for safely erasing transaction data from the local storage of Bitcoin Core nodes. Erasing nodes continue to operate in tune with the network even when erased transaction outputs become relevant for validating subsequent blocks. Using only our basic proof-of-concept implementation, we are already able to safely comply with a significantly larger range of erasure requests than, to the best of our knowledge, any other full node operator so far.
is a widely used technique for locally erasing older parts of a blockchain, mainly with the goal of reducing storage requirements. While related, our local erasure approach differs in its goal---we erase individual data chunks instead of the whole history before a certain point---and provides solutions for outstanding challenges such as the pruning of data potentially relevant for validating future blocks. The latter challenge is highly relevant in practice as problematic data is often encoded in unspent but potentially spendable transaction outputs @cite_22 .
@cite_22
2238041365 2341029337 2747685395 2022619326
Erasure codes are an integral part of many distributed storage systems aimed at Big Data, since they provide high fault-tolerance for low overheads. However, traditional erasure codes are inefficient on reading stored data in degraded environments (when nodes might be unavailable), and on replenishing lost data (vital for long term resilience). Consequently, novel codes optimized to cope with distributed storage system nuances are vigorously being researched. In this paper, we take an engineering alternative, exploring the use of simple and mature techniques ‐ juxtaposing a standard erasure code with RAID4 like parity. We carry out an analytical study to determine the efficacy of this approach over traditional as well as some novel codes. We build upon this study to design CORE, a general storage primitive that we integrate into HDFS. We benchmark this implementation in a proprietary cluster and in EC2. Our experiments show that compared to traditional erasure codes, CORE uses 50 less bandwidth and is up to 75 faster while recovering a single failed node, while the gains are respectively 15 and 60 for double node failures. With the explosion of data in applications all around us, erasure coded storage has emerged as an attractive alternative to replication because even with significantly lower storage overhead, they provide better reliability against data loss. Reed-Solomon code is the most widely used erasure code because it provides maximum reliability for a given storage overhead and is flexible in the choice of coding parameters that determine the achievable reliability. However, reconstruction time for unavailable data becomes prohibitively long mainly because of network bottlenecks. Some proposed solutions either use additional storage or limit the coding parameters that can be used. In this paper, we propose a novel distributed reconstruction technique, called Partial Parallel Repair (PPR), which divides the reconstruction operation to small partial operations and schedules them on multiple nodes already involved in the data reconstruction. Then a distributed protocol progressively combines these partial results to reconstruct the unavailable data blocks and this technique reduces the network pressure. Theoretically, our technique can complete the network transfer in ⌈(log2(k + 1))⌉ time, compared to k time needed for a (k, m) Reed-Solomon code. Our experiments show that PPR reduces repair time and degraded read time significantly. Moreover, our technique is compatible with existing erasure codes and does not require any additional storage overhead. We demonstrate this by overlaying PPR on top of two prior schemes, Local Reconstruction Code and Rotated Reed-Solomon code, to gain additional savings in reconstruction time. In this paper, we introduce Random Erasing, a new data augmentation method for training the convolutional neural network (CNN). In training, Random Erasing randomly selects a rectangle region in an image and erases its pixels with random values. In this process, training images with various levels of occlusion are generated, which reduces the risk of over-fitting and makes the model robust to occlusion. Random Erasing is parameter learning free, easy to implement, and can be integrated with most of the CNN-based recognition models. Albeit simple, Random Erasing is complementary to commonly used data augmentation techniques such as random cropping and flipping, and yields consistent improvement over strong baselines in image classification, object detection and person re-identification. Code is available at: this https URL Distributed storage systems for large-scale applications typically use replication for reliability. Recently, erasure codes were used to reduce the large storage overhead, while increasing data reliability. A main limitation of off-the-shelf erasure codes is their high-repair cost during single node failure events. A major open problem in this area has been the design of codes that: 1) are repair efficient and 2) achieve arbitrarily high data rates. In this paper, we explore the repair metric of locality, which corresponds to the number of disk accesses required during a single node repair. Under this metric, we characterize an information theoretic tradeoff that binds together the locality, code distance, and storage capacity of each node. We show the existence of optimal locally repairable codes (LRCs) that achieve this tradeoff. The achievability proof uses a locality aware flow-graph gadget, which leads to a randomized code construction. Finally, we present an optimal and explicit LRC that achieves arbitrarily high data rates. Our locality optimal construction is based on simple combinations of Reed-Solomon blocks.