Another shortcoming is that current works about joint learning didn't fully exploit the contextual information of the semantic labels. As far as we know, most of them simply utilized the labels to supervise the predictions of semantic and depth splits, making limited contribution to scene understanding of the network. In [1]}, Chen et al. pointed out that how to obtain the correlation of inter-object and intra-object is crucial for depth estimation. Yu et al. [2]} also argued such context make feature representation more robust for semantic segmentation. Therefore, could we excavate the information of labels more deeply to assist the network modeling such correlation? Moreover, most approaches achieve task interaction through adding pixel-wise features [3]}[4]}, simply sharing encoder commonly [5]} or sharing parameters of convolutional layers [6]}. Although these methods leveraged the correlation between tasks, their ways of fusing features are rough. For example, in [4]}, they fused the features via direct weighted addition and then added them to task-specific features. This simple structure may make the network difficult to learn more useful representations of the shared features.
