This paper presents a novel CNN-based approach for synthesizing high-resolution LiDAR point cloud data. Our approach generates semantically and perceptually realistic results with guidance from specialized loss-functions. First, we utilize a modified per-point loss that addresses missing LiDAR point measurements. Second, we align the quality of our generated output with real-world sensor data by applying a perceptual loss. In large-scale experiments on real-world datasets, we evaluate both the geometric accuracy and semantic segmentation performance using our generated data vs. ground truth. In a mean opinion score testing we further assess the perceptual quality of our generated point clouds. Our results demonstrate a significant quantitative and qualitative improvement in both geometry and semantics over traditional non CNN-based upsampling methods.
recently proposed PU-Net which directly operates on three dimensional point clouds @cite_13 . The up-sampling network learns multilevel features per point and expands the point set via multi-branch convolution units. The expanded feature is then split into a multitude of features, which are then reconstructed to an up-sampled point set. The point set is unordered and forms a generic point cloud. However, for our application, it is important to maintain the ordered point cloud structure provided by LiDAR sensors. First, we are able to apply downstream perceptual algorithms which have been designed for the structured low-resolution data. Second, it is possible to re-use valuable data recordings by up-sampling it to higher resolutions especially when new LiDAR sensors with more layers are introduced to the market. Or in the case when algorithms, like semantic segmentation @cite_21 or stixels @cite_3 , have to be adapted to the higher resolution.
@cite_21 @cite_13 @cite_3
2145720667 2900731076 2963680153 2795835984
In this letter, a novel approach that utilizes the spectrum information (i.e., images) provided in a modern light detection and ranging (LiDAR) sensor is proposed for the registration of multistation LiDAR data sets. First, the conjugate points in the images collected at varied LiDAR stations are extracted through the speedup robust feature technique. Then, by applying the image-object space mapping technique, the 3-D coordinates of the conjugate image points can be obtained. Those identified 3-D conjugate points are then fed into a registration model so that the transformation parameters can be immediately solved using the efficient noniterative solution to linear transformations technique. Based on numerical results from a case study, it has been demonstrated that, by implementing the proposed approach, a fully automatic and reliable registration of multistation LiDAR point clouds can be achieved without the need for any human intervention. Unlike images which are represented in regular dense grids, 3D point clouds are irregular and unordered, hence applying convolution on them can be difficult. In this paper, we extend the dynamic filter to a new convolution operation, named PointConv. PointConv can be applied on point clouds to build deep convolutional networks. We treat convolution kernels as nonlinear functions of the local coordinates of 3D points comprised of weight and density functions. With respect to a given point, the weight functions are learned with multi-layer perceptron networks and density functions through kernel density estimation. The most important contribution of this work is a novel reformulation proposed for efficiently computing the weight functions, which allowed us to dramatically scale up the network and significantly improve its performance. The learned convolution kernel can be used to compute translation-invariant and permutation-invariant convolution on any point set in the 3D space. Besides, PointConv can also be used as deconvolution operators to propagate features from a subsampled point cloud back to its original resolution. Experiments on ModelNet40, ShapeNet, and ScanNet show that deep convolutional neural networks built on PointConv are able to achieve state-of-the-art on challenging semantic segmentation benchmarks on 3D point clouds. Besides, our experiments converting CIFAR-10 into a point cloud showed that networks built on PointConv can match the performance of convolutional networks in 2D images of a similar structure. Learning and analyzing 3D point clouds with deep networks is challenging due to the sparseness and irregularity of the data. In this paper, we present a data-driven point cloud upsampling technique. The key idea is to learn multi-level features per point and expand the point set via a multi-branch convolution unit implicitly in feature space. The expanded feature is then split to a multitude of features, which are then reconstructed to an upsampled point set. Our network is applied at a patch-level, with a joint loss function that encourages the upsampled points to remain on the underlying surface with a uniform distribution. We conduct various experiments using synthesis and scan data to evaluate our method and demonstrate its superiority over some baseline methods and an optimization-based method. Results show that our upsampled points have better uniformity and are located closer to the underlying surfaces. 3D LiDAR scanners are playing an increasingly important role in autonomous driving as they can generate depth information of the environment. However, creating large 3D LiDAR point cloud datasets with point-level labels requires a significant amount of manual annotation. This jeopardizes the efficient development of supervised deep learning algorithms which are often data-hungry. We present a framework to rapidly create point clouds with accurate point-level labels from a computer game. The framework supports data collection from both auto-driving scenes and user-configured scenes. Point clouds from auto-driving scenes can be used as training data for deep learning algorithms, while point clouds from user-configured scenes can be used to systematically test the vulnerability of a neural network, and use the falsifying examples to make the neural network more robust through retraining. In addition, the scene images can be captured simultaneously in order for sensor fusion tasks, with a method proposed to do automatic calibration between the point clouds and captured scene images. We show a significant improvement in accuracy (+9 ) in point cloud segmentation by augmenting the training dataset with the generated synthesized data. Our experiments also show by testing and retraining the network using point clouds from user-configured scenes, the weakness blind spots of the neural network can be fixed.