In this paper, we introduce Pairwise-Regularized Resi- dual-Recursive Networks (PR-RRN), a novel neural-network based model for NRSfM. PR-RRN consists of a Residual-Recursive Network (RRN) and two novel losses: Pairwise Contrastive Loss and Pairwise Consistency Loss. RRN alone can reconstruct the non-rigid shapes accurately, and it is further improved by pairwise losses. RRN contains a shape estimation network and a rotation estimation network, and the shape estimation network is constructed with a novel Residual-Recursive module, which is capable to enhance the reconstruction compared to a standard convolution layer with the same number of parameters. And the rotation estimation network is designed to estimate the camera matrix from the 2D input. Furthermore, two pairwise losses regularize the reconstruction in two different aspects, as shown in Fig. REF . Inspired by recent advances in unsupervised representation learning [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, the proposed Pairwise Contrastive Loss encourages higher similarity between the latent representations of high-rigidity pairs of inputs than low-rigidity pairs. The pairwise rigidity is obtained by a novel measurement minimal singular-value ratio. The Pairwise Consistency Loss enforces the reconstruction to be consistent when the estimated shapes and cameras are exchanged between pairs and reprojected as new inputs. The experimental results show that PR-RRN achieves state-of-the-art reconstruction performance on large-scale human motion and categorical objects datasets.
