Fault-Tolerant Collaborative Inference through the Edge-PRUNE Framework
Collaborative inference has received significant research interest in machine
learning as a vehicle for distributing computation load, reducing latency, as
well as addressing privacy preservation in communications. Recent collaborative
inference frameworks have adopted dynamic inference methodologies such as
early-exit and run-time partitioning of neural networks. However, as machine
learning frameworks scale in the number of inference inputs, e.g., in
surveillance applications, fault tolerance related to device failure needs to
be considered. This paper presents the Edge-PRUNE distributed computing
framework, built on a formally defined model of computation, which provides a
flexible infrastructure for fault tolerant collaborative inference. The
experimental section of this work shows results on achievable inference time
savings by collaborative inference, presents fault tolerant system topologies
and analyzes their cost in terms of execution time overhead.