The nexus between IoT devices motivates the research community to push the DNN computation in close proximity of data source, in order to reduce the inference latency, minimize the network cost, and avoid bandwidth bottlenecks [1]}, [2]}.
Particularly, the trained DNN network is split into segments that are assigned to different helpers.
The DNN partitioning and distribution approaches can be categorized into three classes: The main idea of the first approach is to split the model into two segments, where the shallow part is computed in the data-generating device and the deep layers are offloaded to remote servers, i.e., edge or cloud servers. Therefore, the objective will be to decide the split point depending on the size of intermediate data that is relatively small for deep layers and the latency to transmit it, as done in Neurosurgeon [3]}.
The hierarchical partition is the second strategy proposed in the literature, where the DNN network is split and distributed across cloud, edge servers, and end-devices. The work in [4]} adopted this approach combined with exiting the model at early points so that the computation does not always reach cloud servers.
