This paper introduces a method of structure inspection using mixed-reality headsets to reduce the human effort in reporting accurate inspection information such as fault locations in 3D coordinates. Prior to every inspection, the headset needs to be localized. While external pose estimation and fiducial marker based localization would require setup, maintenance, and manual calibration; marker-free self-localization can be achieved using the onboard depth sensor and camera. However, due to limited depth sensor range of portable mixed-reality headsets like Microsoft HoloLens, localization based on simple point cloud registration (sPCR) would require extensive mapping of the environment. Also, localization based on camera image would face same issues as stereo ambiguities and hence depends on viewpoint. We thus introduce a novel approach to Joint Point Cloud and Image-based Localization (JPIL) for mixed-reality headsets that uses visual cues and headset orientation to register small, partially overlapped point clouds and save significant manual labor and time in environment mapping. Our empirical results compared to sPCR show average 10 fold reduction of required overlap surface area that could potentially save on average 20 minutes per inspection. JPIL is not only restricted to inspection tasks but also can be essential in enabling intuitive human-robot interaction for spatial mapping and scene understanding in conjunction with other agents like autonomous robotic systems that are increasingly being deployed in outdoor environments for applications like structural inspection.
While we have covered the related works in the above text, here we emphasize on few other point cloud registration methods that uses visual cues. Dold @cite_6 uses planar patches from image data to refine a point cloud registration whereas Men . @cite_11 uses hue as fourth dimension (x,y,z,hue) and search for correspondence in 4D space. Similarly, authors of @cite_2 @cite_3 use 2D image features in a tightly coupled framework to direct point cloud registration. These requires accurate calibration between the Lidar scans and camera images and work well for dominantly planer structures without stereo ambiguities. When accounted for errors in sensor calibration, stereo ambiguities and complex 3D environments, local image features tend to fail and thus decrease the robustness due to the their tightly coupled nature.
@cite_3 @cite_2 @cite_6 @cite_11
2159029220 2066886972 2001613026 2276426632
This paper presents methodologies to accelerate the registration of 3D point cloud segments by using hue data from the associated imagery. The proposed variant of the Iterative Closest Point (ICP) algorithm combines both normalized point range data and weighted hue value calculated from RGB data of an image registered 3D point cloud. A k-d tree based nearest neighbor search is used to associated common points in x, y, z, hue 4D space. The unknown rigid translation and rotation matrix required for registration is iteratively solved using Singular Value Decomposition (SVD) method. A mobile robot mounted scanner was used to generate color point cloud segments over a large area. The 4D ICP registration has been compared with typical 3D ICP and numerical results on the generated map segments shows that the 4D method resolves ambiguity in registration and converges faster than the 3D ICP Fusion of 3D airborne laser (LIDAR) data and terrestrial optical imagery can be applied in 3D urban modeling and model up-dating. The most challenging aspect of the fusion procedure is registering the terrestrial optical images on the LIDAR point clouds. In this article, we propose an approach for registering these two different data from different sensor sources. As we use iPhone camera images which are taken in front of the interested urban structure by the application user and the high resolution LIDAR point clouds of the acquired by an airborne laser sensor. After finding the photo capturing position and orientation from the iPhone photograph metafile, we automatically select the area of interest in the point cloud and transform it into a range image which has only grayscale intensity levels according to the distance from the image acquisition position. We benefit from local features for registering the iPhone image to the generated range image. In this article, we have applied the registration process based on local feature extraction and graph matching. Finally, the registration result is used for facade texture mapping on the 3D building surface mesh which is generated from the LIDAR point cloud. Our experimental results indicate possible usage of the proposed algorithm framework for 3D urban map updating and enhancing purposes. This paper reports a novel mutual information (MI) based algorithm for automatic registration of unstructured 3D point clouds comprised of co-registered 3D lidar and camera imagery. The proposed method provides a robust and principled framework for fusing the complementary information obtained from these two different sensing modalities. High-dimensional features are extracted from a training set of textured point clouds (scans) and hierarchical k-means clustering is used to quantize these features into a set of codewords. Using this codebook, any new scan can be represented as a collection of codewords. Under the correct rigid-body transformation aligning two overlapping scans, the MI between the codewords present in the scans is maximized. We apply a James-Stein-type shrinkage estimator to estimate the true MI from the marginal and joint histograms of the codewords extracted from the scans. Experimental results using scans obtained by a vehicle equipped with a 3D laser scanner and an omnidirectional camera are used to validate the robustness of the proposed algorithm over a wide range of initial conditions. We also show that the proposed method works well with 3D data alone. Abstract. Registration of point clouds is a necessary step to obtain a complete overview of scanned objects of interest. The majority of the current registration approaches target the general case where a full range of the registration parameters search space is assumed and searched. It is very common in urban objects scanning to have leveled point clouds with small roll and pitch angles and with also a small height differences. For such scenarios the registration search problem can be handled faster to obtain a coarse registration of two point clouds. In this paper, a fully automatic approach is proposed for registration of approximately leveled point clouds. The proposed approach estimates a coarse registration based on three registration parameters and then conducts a fine registration step using iterative closest point approach. The approach has been tested on three data sets of different areas and the achieved registration results validate the significance of the proposed approach.