The second category is based on redundancy and it establishes resilience by assigning each gradient task to more than one node [1]}, [2]}, [3]}, [4]}, [5]}. Existing techniques are sometimes combined with robust aggregation [2]}. Fundamentally, these methods require higher computation load per worker but they come with stronger guarantees of correcting the erroneous gradients. It is important to note that most schemes in this category can be made to fail by a powerful omniscient adversary which can mount judicious attacks [5]}. DRACO [3]} builds on [9]}, [10]} and uses majority vote and Fourier decoders to filter out the adversarial effects. With \(q\)  Byzantines, it guarantees recovery as if the system had no adversaries, when each task is replicated \(r \ge 2q+1\)  times; it is not applicable if this bound is violated. Nonetheless, this requirement is very restrictive for the common assumption that up to half of the workers can be Byzantine. DETOX in [2]} extends DRACO and performs multiple stages of gradient filtering requiring smaller redundancy. However, its resilience guarantees depend heavily on a “random choice” of the adversaries. Subsequent work in [5]} has crafted simple attacks to make this aggregator fail under a more careful choice of adversaries. ByzShield in [5]} borrows ideas from design theory to redundantly assign the tasks in an effort to minimize the distortion fraction.
