We implement the model using the open source machine learning framework Pytorch on a single Nvidia GTX1080Ti GPU. As for the encoder of CI-Net, we choose the ResNet-50 and ResNet-101 as the candidates and both of them are pretrained on the ImageNet classification task [1]}. The learning rates of the pretrained layers are set to be 10 times smaller than the other layers. To avoid the overfitting problem, we adopt the data augmentation strategies, including random rotation, random scaling, random crop, random horizontal flip and random color jitter. The Optimization Algorithm we used is stochastic gradient descent (SGD) [2]} where we set the momentum as 0.9 and the weight decay as \(5e^{-4}\) . To guarantee computational efficiency and fully optimizing the network, we choose the number of set \(\Lambda \)  as 500 to compute the pair loss \(\mathcal {L}_{pair}\) . The weight coefficients \((\alpha ,\beta ,\gamma ,\lambda ,\mu )\)  are set to \((1,5,0.3,1,5)\)  respectively. The training process is divided into three stages. At first, we replace the SUM module with the ground truth attention maps and the model is trained using only \(\mathcal {L}_{depth}\)  and \(\mathcal {L}_{seg}\)  (epochs and learning rates are \((300,6e^{-4})\)  for NYU-Depth-v2 and \((60,6e^{-4})\)  for SUN-RGBD). During the second stage, the SUM is added to the model and \(L_{att}\)  participates in the training process (epochs and learning rates are \((200,2e^{-4})\)  for NYU-Depth-v2 and \((40,3e^{-4})\)  for SUN-RGBD). In the last stage, we employ all the loss costs to train the entire model (we use the polynomical decay strategy with decayed power of 0.9, epochs and initial learning rates are \((200,2e^{-4})\)  for NYU-Depth-v2 and \((40,3e^{-4})\)  for SUN-RGBD).
