The above-mentioned approaches cover mainly the offloading of part of DNN tasks from data-generating devices to high-performance servers, in order to speed up the inference while minimizing the traffic load. Still, in case of high classification load, network congestion and bottleneck can occur. To cater with remote transmission, researchers have started to investigate the feasibility of distributing CNN networks among resource-limited IoT devices in order to jointly compute the inferences locally.
The works in [1]}, [2]}, [3]} presented per-layer CNN partitioning strategy and formulated the allocation approach to distribute different segments into resource constrained devices, as an optimization problem. In addition to per layer partitioning, the DNN can also be divided along the input dimension (e.g., rows or columns of feature maps). Such input wise splitting enables more flexible and fine grained allocation as the data size of each segment can be chosen arbitrarily to be adequate to lightweight devices (e.g., sensors), compared to the fixed partition sizes defined by the layers of the DNN.
However, a major shortcoming of this fine-grained partition is the high dependency between participants. Among the contributions that led the research to examine the feasibility of input-wise partitioning, we can cite DeepThings [4]} and MoDNN [5]}. These efforts studied different DNN splitting scenarios and mainly focused on finding the optimal partitioning that reduces the dependency and data sharing between participants. Still, designing a joint strategy where participants are scheduled to compute different segments of online inference requests while being constrained by their limited resources was not considered by previous works.
