where \(s_{ij}\)  denotes the semantic label of ground truth. \(\mathbf {fd}_{i,j}(l)\)  is the depth feature which has an offset of \(l\)  along \(x\)  or \(y\)  direction and \(\mathbf {fs}_{i,j}(l)\)  is the semantic feature. We use the exponential form of Mahalanobis Distance to measure the discrepancies between features where the covariance matrix \(\mathbf {\Sigma }_{\mathbf {fd}}\)  is set as a diagonal matrix \(\sigma ^{2}\mathbf {I}_{C}\) . Here \(\sigma \)  is a learned parameter from each feature map. Considering that the depth features at the inner edges vary widely while the semantic representations are similar, we weight \(L_{con}\)  by the function \(\psi (\mathchoice{\mathbin {\hbox{\scalebox {.5}{$\m@th \displaystyle \bullet $}}}}{}{}{}\) )\( which returns \) 1\( when the corresponding labels are different and \) 0\( otherwise. Since it is not realistic to take account of all the feature relationships, we select \) l\( from the set \) {1,2}\( so that each feature would be compared for \) 8\( times, which has adequately good performance in our experiments.\)Depth Loss: The depth loss is composed of three items \(\mathcal {L}_{berhu}\) , \(\mathcal {L}_{pair}\)  and \(\mathcal {L}_{norm}\) .The \(\mathcal {L}_{berhu}\)  represents the BerHu Loss providing a good balance of the L1 norm and L2 norm, which is effective in the occasion errors following a heavy-tailed distribution [1]}. The \(\mathcal {L}_{berhu}\)  is defined by
\(\mathcal {L}_{berhu}=\sum _{i,j}{\left\lbrace \begin{array}{ll}\vert d_{i,j}-\tilde{d}_{i,j} \vert & \text{if} \vert d_{i,j}-\tilde{d}_{i,j}\vert \le c,\\\frac{(d_{i,j}-\tilde{d}_{i,j})^{2}+c^{2}}{2c} & \text{if} \vert d_{i,j}-\tilde{d}_{i,j}\vert > c\end{array}\right.},\) 
