This paper revisits the problem of continuous-time structure from motion, and introduces a number of extensions that improve convergence and efficiency. The formulation with a @math -continuous spline for the trajectory naturally incorporates inertial measurements, as derivatives of the sought trajectory. We analyse the behaviour of split interpolation on @math and on @math , and a joint interpolation on @math , and show that the latter implicitly couples the direction of translation and rotation. Such an assumption can make good sense for a camera mounted on a robot arm, but not for hand-held or body-mounted cameras. Our experiments show that split interpolation on @math and on @math is preferable over @math interpolation in all tested cases. Finally, we investigate the problem of landmark reprojection on rolling shutter cameras, and show that the tested reprojection methods give similar quality, while their computational load varies by a factor of 2.
In robotic animation it is instead common to do direct interpolation on the special Euclidean group @math . Recently, such a direct interpolation on @math was applied to the continuous-time structure from motion problem, by integrating the @math spline into an optimization framework . This formulation generalizes the orientation interpolation of @cite_10 to @math . Several recent continuous-time structure from motion papers use the @math approach , while others use separate interpolation of pose and orientation . In the following sections, we analyse the two approaches theoretically, and also compare them experimentally.
@cite_10
1960801737 2066472867 2200993670 2067502459
The author describes a new method for camera autocalibration and scaled Euclidean structure and motion, from three or more views taken by a moving camera with fixed but unknown intrinsic parameters. The motion constancy of these is used to rectify an initial projective reconstruction. Euclidean scene structure is formulated in terms of the absolute quadric-the singular dual 3D quadric (4 spl times 4 rank 3 matrix) giving the Euclidean dot-product between plane normals. This is equivalent to the traditional absolute conic but simpler to use. It encodes both affine and Euclidean structure, and projects very simply to the dual absolute image conic which encodes camera calibration. Requiring the projection to be constant gives a bilinear constraint between the absolute quadric and image conic, from which both can be recovered nonlinearly from m spl ges 3 images, or quasi-linearly from m spl ges 4. Calibration and Euclidean structure follow easily. The nonlinear method is stabler, faster, more accurate and more general than the quasi-linear one. It is based on a general constrained optimization technique-sequential quadratic programming-that may well be useful in other vision problems. In this paper, we propose a new framework to perform motion compression for time-dependent 3D geometric data. Temporal coherence in dynamic geometric models can be used to achieve significant compression, thereby leading to efficient storage and transmission of large volumes of 3D data. The displacement of the vertices in the geometric models is computed using the iterative closest point (ICP) algorithm. This forms the core of our motion prediction technique and is used to estimate the transformation between two successive 3D data sets. The motion between frames is coded in terms of a few affine parameters with some added residues. Our motion segmentation approach separates the vertices into two groups. Within the first group, motion can be encoded with a few affine parameters without the need of residues. In the second group, the vertices need further encoding of residual errors. Also in this group, for those vertices associated with large residual errors under affine mapping, we encode their motion effectively using Newtonian motion estimates. This automatic segmentation enables our algorithm to he very effective in compressing time-dependent geometric data. Dynamic range data captured from the real world, as well as complex animations created using commercial tools, can be compressed efficiently using this scheme. We propose a dense continuous-time tracking and mapping method for RGB-D cameras. We parametrize the camera trajectory using continuous B-splines and optimize the trajectory through dense, direct image alignment. Our method also directly models rolling shutter in both RGB and depth images within the optimization, which improves tracking and reconstruction quality for low-cost CMOS sensors. Using a continuous trajectory representation has a number of advantages over a discrete-time representation (e.g. camera poses at the frame interval). With splines, less variables need to be optimized than with a discrete representation, since the trajectory can be represented with fewer control points than frames. Splines also naturally include smoothness constraints on derivatives of the trajectory estimate. Finally, the continuous trajectory representation allows to compensate for rolling shutter effects, since a pose estimate is available at any exposure time of an image. Our approach demonstrates superior quality in tracking and reconstruction compared to approaches with discrete-time or global shutter assumptions. A common motion interpolation technique for realistic human animation is to blend similar motion samples with weighting functions whose parameters are embedded in an abstract space. Existing methods, however, are insensitive to statistical properties, such as correlations between motions. In addition, they lack the capability to quantitatively evaluate the reliability of synthesized motions. This paper proposes a method that treats motion interpolations as statistical predictions of missing data in an arbitrarily definable parametric space. A practical technique of geostatistics, called universal kriging, is then introduced for statistically estimating the correlations between the dissimilarity of motions and the distance in the parametric space. Our method statistically optimizes interpolation kernels for given parameters at each frame, using a pose distance metric to efficiently analyze the correlation. Motions are accurately predicted for the spatial constraints represented in the parametric space, and they therefore have few undesirable artifacts, if any. This property alleviates the problem of spatial inconsistencies, such as foot-sliding, that are associated with many existing methods. Moreover, numerical estimates for the reliability of predictions enable motions to be adaptively sampled. Since the interpolation kernels are computed with a linear system in real-time, motions can be interactively edited using various spatial controls.