Consistency Loss: Inspired by [1]}, we design a consistency loss to make semantic and depth branches guide each other mutually. Specifically, the features that are distinct or similar in semantic feature map should keep the same nature as in depth representations. For example, the semantic features of the sky and tree are extremely different since they belong to different classes, while the corresponding two depth features are also discrepant because the distances of them have large gap. Therefore, we employ this character to supervise the consistency loss of task-specific features, the form of which is defined as
\(\begin{split}&\mathcal {L}_{con}=\sum _{l}\sum _{i,j}\psi (s_{i,j}, s_{i,j}(l))\vert D(\mathbf {fd}_{i,j},l) - D(\mathbf {fs}_{i,j},l)\vert \\&D(\mathbf {fd}_{i,j},l)=\exp [-\frac{1}{2}(\mathbf {fd}_{i,j}\!-\!\mathbf {fd}_{i,j}(l))^{T}\mathbf {\Sigma }^{-1}_\mathbf {fd}(\mathbf {fd}_{i,j}\!-\!\mathbf {fd}_{i,j}(l))].\end{split}\) 
