Many countermeasures have been designed to enhance the privacy of distributed deep networks. One of these measures is the deep split [1]}, where the model is split and distributed after fully-connected (fc) layers. Particularly, owing to their ability to mix the input and hide the sample features, the outputs of fully-connected layers are difficult to be recovered and the initial data cannot be generated. Thus, the idea is to compute the first segment in the data-generating device, while delegating the second part to another participant. However, following this approach, all convolutional layers should be hosted in the source device, which is not always possible and compatible with all IoT devices (e.g., sensors.) due to the complexity and resource requirements of these tasks.
A second approach [2]} proposes to extract the features
from the original image or from one of the layers' outputs using an encoder and transmit the data to the centralized server for inference. This approach prevents the exposure of irrelevant information to the untrusted party that may use it for unwanted inferences. However, in some cases, the amount of extracted features can be sufficient for adversary to recover the original sample. Whereas, less may also result in low classification accuracy. We highlight also that the large amount of extracted data offloaded to remote servers may become challenging, especially for systems with unstable bandwidth availability.
