Place recognition and loop-closure detection are main challenges in the localization, mapping and navigation tasks of self-driving vehicles. In this paper, we solve the loop-closure detection problem by incorporating the deep-learning based point cloud description method and the coarse-to-fine sequence matching strategy. More specifically, we propose a deep neural network to extract a global descriptor from the original large-scale 3D point cloud, then based on which, a typical place analysis approach is presented to investigate the feature space distribution of the global descriptors and select several super keyframes. Finally, a coarse-to-fine strategy, which includes a super keyframe based coarse matching stage and a local sequence matching stage, is presented to ensure the loop-closure detection accuracy and real-time performance simultaneously. Thanks to the sequence matching operation, the proposed approach obtains an improvement against the existing deep-learning based methods. Experiment results on a self-driving vehicle validate the effectiveness of the proposed loop-closure detection algorithm.
In order to solve the above problems, deep neural network was introduced for 3D point cloud feature learning and achieved state-of-the-art performance. Some work attempts to convert point cloud input to a regular 3D volume representation to alleviate the orderless problem of the point cloud, such as the volumetric CNNs @cite_20 and 3D ShapeNets @cite_15 . They are suitable for point cloud-based classification and recognition, respectively. On the other hand, different from volumetric representation, Multiview CNNs @cite_16 projects the 3D data into 2D images so that 2D CNN can be performed. Additionally, by achieving the permutation invariance in the network, PointNet @cite_7 makes it possible to learn features directly from the raw point cloud data. Although PointNet has achieved superior performance on small-scale shape classification and recognition tasks, it did not scale well for large-scale place recognition problem. PointNetVLAD @cite_23 is the first work that directly applies 3D point cloud to large-scale place recognition, but this method dose not consider local feature extraction adequately, and the network is heavy in memory when the feature dimension is huge.
@cite_7 @cite_23 @cite_15 @cite_16 @cite_20
2734558172 2896229066 2963158438 2624503621
In the last few years, Convolutional Neural Networks (CNNs) had become the default paradigm to address classification problems, specially, but not only, in image recognition. This is mainly due to the high success rate that they provide. Despite there currently exist approaches that apply deep learning to the 3D recognition problem, they are either too slow for online uses or too error prone. To fill this gap, we propose LonchaNet, a deep learning architecture for point clouds classification. Our system successfully achieves a high accuracy yet providing a low computation cost. A dense set of experiments were carried out in order to validate our system in the frame of the ModelNet — a large-scale 3D CAD models dataset — challenge. Our proposal achieves a success rate of 94.37 in the ModelNet-10 classification task, the second place in the leaderboard as of today (November, 2016). Convolutional Neural Networks (CNNs)-based methods for 3D hand pose estimation with depth cameras usually take 2D depth images as input and directly regress holistic 3D hand pose. Different from these methods, our proposed Point-to-Point Regression PointNet directly takes the 3D point cloud as input and outputs point-wise estimations, i.e., heat-maps and unit vector fields on the point cloud, representing the closeness and direction from every point in the point cloud to the hand joint. The point-wise estimations are used to infer 3D joint locations with weighted fusion. To better capture 3D spatial information in the point cloud, we apply a stacked network architecture for PointNet with intermediate supervision, which is trained end-to-end. Experiments show that our method can achieve outstanding results when compared with state-of-the-art methods on three challenging hand pose datasets. Deep neural networks have enjoyed remarkable success for various vision tasks, however it remains challenging to apply CNNs to domains lacking a regular underlying structures such as 3D point clouds. Towards this we propose a novel convolutional architecture, termed SpiderCNN, to efficiently extract geometric features from point clouds. SpiderCNN is comprised of units called SpiderConv, which extend convolutional operations from regular grids to irregular point sets that can be embedded in ( R ^n ), by parametrizing a family of convolutional filters. We design the filter as a product of a simple step function that captures local geodesic information and a Taylor polynomial that ensures the expressiveness. SpiderCNN inherits the multi-scale hierarchical architecture from classical CNNs, which allows it to extract semantic deep features. Experiments on ModelNet40 demonstrate that SpiderCNN achieves state-of-the-art accuracy (92.4 ) on standard benchmarks, and shows competitive performance on segmentation task. Few prior works study deep learning on point sets. PointNet by is a pioneer in this direction. However, by design PointNet does not capture local structures induced by the metric space points live in, limiting its ability to recognize fine-grained patterns and generalizability to complex scenes. In this work, we introduce a hierarchical neural network that applies PointNet recursively on a nested partitioning of the input point set. By exploiting metric space distances, our network is able to learn local features with increasing contextual scales. With further observation that point sets are usually sampled with varying densities, which results in greatly decreased performance for networks trained on uniform densities, we propose novel set learning layers to adaptively combine features from multiple scales. Experiments show that our network called PointNet++ is able to learn deep point set features efficiently and robustly. In particular, results significantly better than state-of-the-art have been obtained on challenging benchmarks of 3D point clouds.