Depth-based Perceptual Similarity.
We extend the concept of Perceptual Similarity metric of [1]} from images to depth-related data (Depth-only or RGBD).
Specifically, to compare a reconstructed depth map \(\hat{s}\)  to a target depth map \(s\) , we first feed both into the trained Depth-only recognition network, \(\varphi _D\) , and extract features from multiple blocks (from low to high layers, corresponding to lower-to-higher “semantic” levels).
Similarly, when \(s\)  and \(\hat{s}\)  are RGBD data, we use the trained RGBD-based recognition network, \(\varphi _{RGBD}\) .
For brevity we henceforth refer to both networks as \(\varphi \) .
We denote the deep features extracted from an input \(s\) 
at the output of a particular block \(b\) , by \(\varphi ^{b}\left(s\right)\) .
The perceptual similarity between \(s\)  and \(\hat{s}\) , \(\mathcal {L}_{perceptual}\left(\hat{s},s\right)\) , is then defined by cosine similarity between channel-normalized ground-truth and predicted features at each block output:
