In this paper we present a semi-automatic 2D-3D local registration pipeline capable of coloring 3D models obtained from 3D scanners by using uncalibrated images. The proposed pipeline exploits the Structure from Motion (SfM) technique in order to reconstruct a sparse representation of the 3D object and obtain the camera parameters from image feature matches. We then coarsely register the reconstructed 3D model to the scanned one through the Scale Iterative Closest Point (SICP) algorithm. SICP provides the global scale, rotation and translation parameters, using minimal manual user intervention. In the final processing stage, a local registration refinement algorithm optimizes the color projection of the aligned photos on the 3D object removing the blurring ghosting artefacts introduced due to small inaccuracies during the registration. The proposed pipeline is capable of handling real world cases with a range of characteristics from objects with low level geometric features to complex ones.
Nevertheless there are 3D scanners which provide also reflectance images and the registration is performed in a 2D space @cite_19 , @cite_25 . On the other hand, some authors perform their registration in a 3D space reconstructing the 3D object from the 2D images and aligning both 3D objects @cite_28 , @cite_11 , @cite_3 . This procedure is carried out in two steps: (1) 3D reconstruction and (2) point cloud alignment. Through the widely used Structure from Motion technique (SfM), a 3D reconstruction and intrinsic and extrinsic camera parameters are recovered without making any assumptions about the images in the scene. The registration is usually performed by selecting correspondences @cite_11 that minimize the distances between a set of points.
@cite_28 @cite_3 @cite_19 @cite_25 @cite_11
2002500614 2001613026 2108841317 2028525317
The photorealistic acquisition of 3D objects often requires color information from digital photography to be mapped on the acquired geometry, in order to obtain a textured 3D model. This paper presents a novel fully automatic 2D 3D global registration pipeline consisting of several stages that simultaneously register the input image set on the corresponding 3D object. The first stage exploits Structure From Motion (SFM) on the image set in order to generate a sparse point cloud. During the second stage, this point cloud is aligned to the 3D object using an extension of the 4 Point Congruent Set (4PCS) algorithm for the alignment of range maps. The extension accounts for models with different scales and unknown regions of overlap. In the last processing stage a global refinement algorithm based on mutual information optimizes the color projection of the aligned photos on the 3D object, in order to obtain high quality textures. The proposed registration pipeline is general, capable of dealing with small and big objects of any shape, and robust. We present results from six real cases, evaluating the quality of the final colors mapped onto the 3D object. A comparison with a ground truth dataset is also presented. This paper reports a novel mutual information (MI) based algorithm for automatic registration of unstructured 3D point clouds comprised of co-registered 3D lidar and camera imagery. The proposed method provides a robust and principled framework for fusing the complementary information obtained from these two different sensing modalities. High-dimensional features are extracted from a training set of textured point clouds (scans) and hierarchical k-means clustering is used to quantize these features into a set of codewords. Using this codebook, any new scan can be represented as a collection of codewords. Under the correct rigid-body transformation aligning two overlapping scans, the MI between the codewords present in the scans is maximized. We apply a James-Stein-type shrinkage estimator to estimate the true MI from the marginal and joint histograms of the codewords extracted from the scans. Experimental results using scans obtained by a vehicle equipped with a 3D laser scanner and an omnidirectional camera are used to validate the robustness of the proposed algorithm over a wide range of initial conditions. We also show that the proposed method works well with 3D data alone. We present a method for fusing two acquisition modes, 2D photographs and 3D LiDAR scans, for depth-layer decomposition of urban facades. The two modes have complementary characteristics: point cloud scans are coherent and inherently 3D, but are often sparse, noisy, and incomplete; photographs, on the other hand, are of high resolution, easy to acquire, and dense, but view-dependent and inherently 2D, lacking critical depth information. In this paper we use photographs to enhance the acquired LiDAR data. Our key observation is that with an initial registration of the 2D and 3D datasets we can decompose the input photographs into rectified depth layers. We decompose the input photographs into rectangular planar fragments and diffuse depth information from the corresponding 3D scan onto the fragments by solving a multi-label assignment problem. Our layer decomposition enables accurate repetition detection in each planar layer, using which we propagate geometry, remove outliers and enhance the 3D scan. Finally, the algorithm produces an enhanced, layered, textured model. We evaluate our algorithm on complex multi-planar building facades, where direct autocorrelation methods for repetition detection fail. We demonstrate how 2D photographs help improve the 3D scans by exploiting data redundancy, and transferring high level structural information to (plausibly) complete large missing regions. Abstract. Three-dimensional (3D) models of historical buildings are created for documentation and virtual realization of them. Laser scanning and photogrammetry are extensively used to perform for these aims. The selection of the method that will be used in threedimensional modelling study depends on the scale and shape of the object, and also applicability of the method. Laser scanners are high cost instruments. However, the cameras are low cost instruments. The off-the-shelf cameras are used for taking the photogrammetric images. The camera is imaging the object details by carrying on hand while the laser scanner makes ground based measurement. Laser scanner collect high density spatial data in a short time from the measurement area. On the other hand, image based 3D (IB3D) measurement uses images to create 3D point cloud data. The image matching and the creation of the point cloud can be done automatically. Historical buildings include more complex details. Thus, all details cannot be measured by terrestrial laser scanner (TLS) due to the blocking the details with each others. Especially, the artefacts which have complex shapes cannot be measured in full details. They cause occlusion on the point cloud model. However it is possible to record photogrammetric images and creation IB3D point cloud for these areas. Thus the occlusion free 3D model is created by the integration of point clouds originated from the TLS and photogrammetric images. In this study, usability of laser scanning in conjunction with image based modelling for creation occlusion free three-dimensional point cloud model of historical building was evaluated. The IB3D point cloud was created in the areas that could not been measured by TLS. Then laser scanning and IB3D point clouds were integrated in the common coordinate system. The registration point clouds were performed with the iterative closest point (ICP) and georeferencing methods. Accuracy of the registration was evaluated by convergency and its standard deviations for the ICP and residuals on the control points for the georeferencing method.