This paper introduces a method of structure inspection using mixed-reality headsets to reduce the human effort in reporting accurate inspection information such as fault locations in 3D coordinates. Prior to every inspection, the headset needs to be localized. While external pose estimation and fiducial marker based localization would require setup, maintenance, and manual calibration; marker-free self-localization can be achieved using the onboard depth sensor and camera. However, due to limited depth sensor range of portable mixed-reality headsets like Microsoft HoloLens, localization based on simple point cloud registration (sPCR) would require extensive mapping of the environment. Also, localization based on camera image would face same issues as stereo ambiguities and hence depends on viewpoint. We thus introduce a novel approach to Joint Point Cloud and Image-based Localization (JPIL) for mixed-reality headsets that uses visual cues and headset orientation to register small, partially overlapped point clouds and save significant manual labor and time in environment mapping. Our empirical results compared to sPCR show average 10 fold reduction of required overlap surface area that could potentially save on average 20 minutes per inspection. JPIL is not only restricted to inspection tasks but also can be essential in enabling intuitive human-robot interaction for spatial mapping and scene understanding in conjunction with other agents like autonomous robotic systems that are increasingly being deployed in outdoor environments for applications like structural inspection.
While we have covered the related works in the above text, here we emphasize on few other point cloud registration methods that uses visual cues. Dold @cite_6 uses planar patches from image data to refine a point cloud registration whereas Men . @cite_11 uses hue as fourth dimension (x,y,z,hue) and search for correspondence in 4D space. Similarly, authors of @cite_2 @cite_3 use 2D image features in a tightly coupled framework to direct point cloud registration. These requires accurate calibration between the Lidar scans and camera images and work well for dominantly planer structures without stereo ambiguities. When accounted for errors in sensor calibration, stereo ambiguities and complex 3D environments, local image features tend to fail and thus decrease the robustness due to the their tightly coupled nature.
@cite_3 @cite_2 @cite_6 @cite_11
2142008986 2145720667 2922018968 2159029220
Building 3D models using terrestrial laser scanner (TLS) data is currently an active area of research, especially in the fields of heritage recording and site documentation. Multiple TLS scans are often required to generate an occlusion-free 3D model in situations where the object to be recorded has a complex geometry. The first task associated with building 3D models from laser scanner data in such cases is to transform the data from the scanner’s local coordinate system into a uniform Cartesian reference datum, which requires sufficient overlap between the scans. Many TLS systems are now supplied with an SLR-type digital camera, such that the scene to be scanned can also be photographed. The provision of overlapping imagery offers an alternative, photogrammetric means to achieve point cloud registration between adjacent scans. The images from the digital camera mounted on top of the laser scanner are used to first relatively orient the network of images, and then to transfer this orientation to the TLS stations to provide exterior orientation. The proposed approach, called the IBR method for Image-Based Registration, offers a one-step registration of the point clouds from each scanner position. In the case of multiple scans, exterior orientation is simultaneously determined for all TLS stations by bundle adjustment. This paper outlines the IBR method and discusses test results obtained with the approach. It will be shown that the photogrammetric orientation process for TLS point cloud registration is efficient and accurate, and offers a viable alternative to other approaches, such as the well-known iterative closest point algorithm. In this letter, a novel approach that utilizes the spectrum information (i.e., images) provided in a modern light detection and ranging (LiDAR) sensor is proposed for the registration of multistation LiDAR data sets. First, the conjugate points in the images collected at varied LiDAR stations are extracted through the speedup robust feature technique. Then, by applying the image-object space mapping technique, the 3-D coordinates of the conjugate image points can be obtained. Those identified 3-D conjugate points are then fed into a registration model so that the transformation parameters can be immediately solved using the efficient noniterative solution to linear transformations technique. Based on numerical results from a case study, it has been demonstrated that, by implementing the proposed approach, a fully automatic and reliable registration of multistation LiDAR point clouds can be achieved without the need for any human intervention. This paper presents methodologies to accelerate the registration of 3D point cloud segments by using hue data from the associated imagery. The proposed variant of the Iterative Closest Point (ICP) algorithm combines both normalized point range data and weighted hue value calculated from RGB data of an image registered 3D point cloud. A k-d tree based nearest neighbor search is used to associated common points in x, y, z, hue 4D space. The unknown rigid translation and rotation matrix required for registration is iteratively solved using Singular Value Decomposition (SVD) method. A mobile robot mounted scanner was used to generate color point cloud segments over a large area. The 4D ICP registration has been compared with typical 3D ICP and numerical results on the generated map segments shows that the 4D method resolves ambiguity in registration and converges faster than the 3D ICP