In Figure REF , we plot the depth error in the form of Abs Rel and RMSE(log) for both the first stage and the final output. To demonstrate the advantage of end-to-end training, we add the results from monocular depth estimation with CoordNet [1]} as backbone (stand-alone CoordNet) to the plot. The experiment is conducted on Omnidirectional Stereo Dataset [1]} where the left-down view is used for training and testing.
We can see that the coarse depth estimation network converges fast at earlier epochs. This enables the second stage stereo matching network to converge. Moreover, since our pipeline is en-to-end, the gradient of the second stage flows back to the view synthesis stage and facilitates the generation of better quality novel synthesized views. We compare a stand-alone CoordNet with the CoordNet that is integrated in our full pipeline. As shown in Figure REF , the stand-alone version has the largest depth prediction error, which indicates that simultaneous training for networks in two stage is beneficial for convergence.
