Many man-made objects have intrinsic symmetries and Manhattan structure. By assuming an orthographic projection model, this paper addresses the estimation of 3D structures and camera projection using symmetry and or Manhattan structure cues, which occur when the input is single- or multiple-image from the same category, e.g., multiple different cars. Specifically, analysis on the single image case implies that Manhattan alone is sufficient to recover the camera projection, and then the 3D structure can be reconstructed uniquely exploiting symmetry. However, Manhattan structure can be difficult to observe from a single image due to occlusion. To this end, we extend to the multiple-image case which can also exploit symmetry but does not require Manhattan axes. We propose a novel rigid structure from motion method, exploiting symmetry and using multiple images from the same category as input. Experimental results on the Pascal3D+ dataset show that our method significantly outperforms baseline methods.
Symmetry has been studied in computer vision for several decades. For example, symmetry has been used as a cue in depth recovery @cite_10 @cite_31 @cite_24 as well as for recognizing symmetric objects @cite_44 . Grossmann and Santos-Victor utilized various geometric clues, such as planarity, orthogonality, parallelism and symmetry, for 3D scene reconstruction @cite_29 @cite_25 , where the camera rotation matrix was pre-computed by vanishing points @cite_40 . Recently, researchers applied symmetry to scene reconstruction @cite_26 , and 3D mesh reconstruction with occlusion @cite_9 . In addition, symmetry, incorporated with planarity and compactness priors, has also been studied to reconstruct structures defined by 3D keypoints @cite_38 . By contrast, the Manhattan world assumption was developed originally for scenes @cite_19 @cite_20 @cite_21 , where the authors assumed visual scenes were based on a Manhattan 3D grid which provided 3 perpendicular axis constaints. Both symmetry and Manhattan can be straightforwardly combined, and adapted to 3D object reconstruction, particularly for man made objects.
@cite_38 @cite_26 @cite_29 @cite_9 @cite_21 @cite_24 @cite_44 @cite_40 @cite_19 @cite_31 @cite_10 @cite_25 @cite_20
2171543226 2081709959 1973016074 2029528398 2139018239 2102271310 2009751730 2143450077 1986377419
We present a new algorithm for reconstructing 3D shapes. The algorithm takes one 2D image of a 3D shape and reconstructs the 3D shape by applying a priori constraints: symmetry, planarity and compactness. The shape is reconstructed without using information about the surfaces, such as shading, texture, binocular disparity or motion. Performance of the algorithm is illustrated on symmetric polyhedra, but the algorithm can be applied to a very wide range of shapes. Psychophysical plausibility of the algorithm is discussed. In this paper, we provide a principled explanation of how knowledge in global 3-D structural invariants, typically captured by a group action on a symmetric structure, can dramatically facilitate the task of reconstructing a 3-D scene from one or more images. More importantly, since every symmetric structure admits a “canonical” coordinate frame with respect to which the group action can be naturally represented, the canonical pose between the viewer and this canonical frame can be recovered too, which explains why symmetric objects (e.g., buildings) provide us overwhelming clues to their orientation and position. We give the necessary and sufficient conditions in terms of the symmetry (group) admitted by a structure under which this pose can be uniquely determined. We also characterize, when such conditions are not satisfied, to what extent this pose can be recovered. We show how algorithms from conventional multiple-view geometry, after properly modified and extended, can be directly applied to perform such recovery, from all “hidden images” of one image of the symmetric structure. We also apply our results to a wide range of applications in computer vision and image processing such as camera self-calibration, image segmentation and global orientation, large baseline feature matching, image rendering and photo editing, as well as visual illusions (caused by symmetry if incorrectly assumed). We investigate the constraints placed on the image projection of a planar object having local reflectional symmetry. Under the affine approximation to projection, we demonstrate an efficient (low-complexity) algorithm for detecting and verifying symmetries despite the distorting effects of image skewing. The symmetries are utilized for three distinct tasks: first, determining image back-projection up to a similarity transformation ambiguity; second, determining the object plane orientation (slant and tilt); and third, as a test for non-coplanarity amongst a collection of objects. These results are illustrated throughout with examples from images of real scenes. According to the 1.5-views theorem (Poggio, Technical Report #9005-03, IRST, Povo, 1990; Ullman and Basri, IEEE Trans. PAMI 13, 992-1006, 1991) recognition of a specific 3D object (defined in terms of pointwise features) from a novel 2D view can be achieved from at least two 2D model views (for each object, for orthographic projection). This note considers how recognition can be achieved from a single 2D model view by exploiting prior knowledge of an object's symmetry. It is proved that, for any bilaterally symmetric 3D object, one non-accidental 2D model view is sufficient for recognition since it can be used to generate additional 'virtual' views. It is also proved that, for bilaterally symmetric objects, the correspondence of four points between two views determines the correspondence of all other points. Symmetries of higher order allow the recovery of Euclidean structure from a single 2D view.1 We present a method for reconstruction of structured scenes from one or more views, in which the user provides image points and geometric knowledge -coplanarity, ratios of distances, angles- about the corresponding 3D points. First, the geometric information is analyzed. Then vanishing points are estimated, from which camera calibration is obtained. Finally, an algebraic method gives the reconstruction. Our algebraic reconstruction method improves the present state-of-the-art in many aspects : geometric knowledge includes not only planarity and alignment information, but also known ratios of lengths. The single and multipleview cases are treated in the same way and the method detects whether the input data is sufficient to define a rigid reconstruction. We benchmark, using synthetic data, the various steps of the estimation process and show reconstructions obtained from real-world situations in which other methods would fail. We also present a new method for maximum likelihood estimation of vanishing points. When designing computer vision systems for the blind and visually impaired it is important to determine the orientation of the user relative to the scene. We observe that most indoor and outdoor (city) scenes are designed on a Manhattan three-dimensional grid. This Manhattan grid structure puts strong constraints on the intensity gradients in the image. We demonstrate an algorithm for detecting the orientation of the user in such scenes based on Bayesian inference using statistics which we have learnt in this domain. Our algorithm requires a single input image and does not involve pre-processing stages such as edge detection and Hough grouping. We demonstrate strong experimental results on a range of indoor and outdoor images. We also show that estimating the grid structure makes it significantly easier to detect target objects which are not aligned with the grid. A new technique dramatically simplifies the analysis of matching and depth reconstruction by extracting three-dimensional rigid depth interpretation from pairwise comparisons of weak perspective projections. This method provides a simple linear criterion for testing the correctness of correspondence for a pair of images; the method also provides a description of a one-parameter family of interpretations for each pair of images that satisfies this criterion. We show that if at least three projections of a volumetric object are known, then a three-dimensional (3D) rigid interpretation can be inferred from pairwise comparisons between any one of these images and other images in the set. The 3D interpretation is derived from the intersection of corresponding one-parameter families. The method provides a common computational basis for different processes of depth perception, for example, depth-from-stereo and depth-from-motion. Thus, a single mechanism for these processes in the human visual system would be sufficient. The proposed method does not require information about relative positions of eye(s) or camera(s) for different projections, but this information can be easily incorporated. The method can be applied for pairwise comparison within a single image. If any nontrivial correspondence is found, then several views of the same object are present in the same image. This happens, for example, in views of volumetrically symmetric objects. Symmetry facilitates depth reconstruction; if an object possesses two or more symmetries, its depth can be reconstructed from a single image. We present a method to reconstruct from one or more images a scene that is rich in planes, alignments, symmetries, orthogonalities, and other forms of geometrical regularity. Given image points of interest and some geometric information, the method recovers least-squares estimates of the 3D points, camera position(s), orientation(s), and eventually calibration(s). Our contributions lie (i) in a novel way of exploiting some types of symmetry and of geometric regularity, (ii) in treating indifferently one or more images, (iii) in a geometric test that indicates whether the input data uniquely defines a reconstruction, and (iv) a parameterization method for collections of 3D points subject to geometric constraints. Moreover, the reconstruction algorithm lends itself to sensitivity analysis. The method is benchmarked on synthetic data and its effectiveness is shown on real-world data. This letter argues that many visual scenes are based on a "Manhattan" three-dimensional grid that imposes regularities on the image statistics. We construct a Bayesian model that implements this assumption and estimates the viewer orientation relative to the Manhattan grid. For many images, these estimates are good approximations to the viewer orientation (as estimated manually by the authors). These estimates also make it easy to detect outlier structures that are unaligned to the grid. To determine the applicability of the Manhattan world model, we implement a null hypothesis model that assumes that the image statistics are independent of any three-dimensional scene structure. We then use the log-likelihood ratio test to determine whether an image satisfies the Manhattan world assumption. Our results show that if an image is estimated to be Manhattan, then the Bayesian model's estimates of viewer direction are almost always accurate (according to our manual estimates), and vice versa.