The conventional methods for estimating camera poses and scene structures from severely blurry or low resolution images often result in failure. The off-the-shelf deblurring or super-resolution methods may show visually pleasing results. However, applying each technique independently before matching is generally unprofitable because this naive series of procedures ignores the consistency between images. In this paper, we propose a pioneering unified framework that solves four problems simultaneously, namely, dense depth reconstruction, camera pose estimation, super-resolution, and deblurring. By reflecting a physical imaging process, we formulate a cost minimization problem and solve it using an alternating optimization technique. The experimental results on both synthetic and real videos show high-quality depth maps derived from severely degraded images that contrast the failures of naive multi-view stereo methods. Our proposed method also produces outstanding deblurred and super-resolved images unlike the independent application or combination of conventional video deblurring, super-resolution methods.
By contrast, our proposed blur model () covers more general camera motions by adopting the linear model in an Lie algebra space @cite_20 . The blur kernel is explicitly approximated by interpolating the camera path and depth maps between adjacent frames. Figure shows the difference between the conventional and proposed blur models.
@cite_20
88434223
An arbitrary rigid transformation in SE(3) can be separated into two parts, namely, a translation and a rigid rotation. This technical report reviews, under a unifying viewpoint, three common alternatives to representing the rotation part: sets of three (yaw-pitch-roll) Euler angles, orthogonal rotation matrices from SO(3) and quaternions. It will be described: (i) the equivalence between these representations and the formulas for transforming one to each other (in all cases considering the translational and rotational parts as a whole), (ii) how to compose poses with poses and poses with points in each representation and (iii) how the uncertainty of the poses (when modeled as Gaussian distributions) is affected by these transformations and compositions. Some brief notes are also given about the Jacobians required to implement least-squares optimization on manifolds, an very promising approach in recent engineering literature. The text reflects which MRPT C++ library1 functions implement each of the described algorithms. All the implementations have been thoroughly validated by means of unit testing and numerical estimation of the Jacobians. http: www.mrpt.org A tutorial on SE(3) transformation parameterizations and on-manifold optimization MAPIR Group Technical report #012010 Dpto. de Ingenieŕia de Sistemas y Automatica http: mapir.isa.uma.es History of document versions: • Version 5: Fixed a typo in Eq. 9.19 (21 Oct 2014) (Thanks to Tanner Schmidt for reporting) • Version 4: Added formulas for the Jacobian of the SO(3) logarithm map, in §10.3.2 (9 May 2013) • Version 3: Added the explicit formulas for the logarithm map of SO(3) and SE(3), fixed error in Eq. (10.25), explained the equivalence between the yaw-pitch-roll and roll-pitch-yaw forms and introduction of the [lnR] ▽ notation when discussing the logarithm maps (14 Aug 2012) • Version 2: Added more Jacobians (§10.3.5, §10.3.6, §10.3.4), the Appendix A and approximation in §10.3.8. (12 Sep 2010) • Version 1: First version (Released 1 Sep 2010). Notice: Part of this report was also published within chapter 10 and appendix IV of the book [6].