NRSfM. The problem of non-rigid structure from motion is first introduced in the research of recovering a sequence of 3D face landmarks and camera positions by Bregler et al. [1]}. This research proposes a widely accepted assumption that the deforming 3D shapes can be compactly represented as a linear combination of a small number of basis shapes. Although this low rank assumption has been made, the deformation of shapes still remains under-constrained, which makes the NRSfM a challenging task for years. Various types of constraints [2]}, [3]}, [4]}, [5]} have been explored to restrict the deforming 3D structure. Xiao et al. [6]} propose a basis constraint to reach a close-form solution of NRSfM factorization. Torresani et al. [7]} develop a Gaussian prior of the shape coefficients and reconstruct the shapes and cameras with probabilistic principal components analysis. With a sequence as the input, temporal smoothness can be leveraged to improve the reconstruction. Akhter et al. [8]} introduce a dual representation of the NRSfM problem.
Gotardo et al. [9]} formulate the temporal deformation of shapes as a smooth trajectory over the coefficients of shape basis, and this idea is improved by modeling the shape trajectory and basis shape in low-dimensional manifold [10]} and using kernel to measure the distance [11]}. There is a milestone that Dai et al. [12]} propose a block matrix method and achieve the outstanding performance with low rank priors. There are more breakthroughs [13]}, [14]}, [15]}, [16]}, [17]} in the field like inextensible [18]}, [5]}, piecewise [20]} methods, metric projection [21]}.
