Service robots are expected to be more autonomous and efficiently work in human-centric environments. For this type of robots, open-ended object recognition is a challenging task due to the high demand for two essential capabilities: (i) the accurate and real-time response, and (ii) the ability to learn new object categories from very few examples on-site. These capabilities are required for such robots since no matter how extensive the training data used for batch learning, the robot might be faced with an unknown object when operating in everyday environments. In this work, we present OrthographicNet, a deep transfer learning based approach, for 3D object recognition in open-ended domains. In particular, OrthographicNet generates a rotation and scale invariant global feature for a given object, enabling to recognize the same or similar objects seen from different perspectives. Experimental results show that our approach yields significant improvements over the previous state-of-the-art approaches concerning scalability, memory usage, and object recognition performance. Regarding real-time performance, two real-world demonstrations validate the promising performance of the proposed architecture. Moreover, our approach demonstrates the capability of learning from very few training examples in a real-world setting.
The pointset-based approaches are completely different from the other two. PointNet proposed by @cite_31 directly takes unordered point sets as inputs. PointNet learns a global representation of a point cloud based on computing individual point features from per-point Multi-Layer-Perceptron (MLP) first and then aggregating all features of the given object. Recently @cite_33 improved PointNet by exploiting local structures induced by the metric space. In particular, PointNet++ segments a point cloud into smaller clusters, and then send each cluster through a small PointNet. Of course, this leads to a complicated architecture with reduced speed and not suitable for real-time application. In another work, Klokov et.al @cite_5 proposed Kd-Networks for the recognition of 3D object represented by 3D point cloud. We compare our method with sate-of-the-art deep learning methods including 3DShapeNet @cite_21 , DeepPano @cite_7 , BeamNet @cite_1 , GeometryImage @cite_16 , and PointNet @cite_31 .
@cite_33 @cite_7 @cite_21 @cite_1 @cite_5 @cite_31 @cite_16
2624503621 2963053547 2796040722 2963121255
Few prior works study deep learning on point sets. PointNet by is a pioneer in this direction. However, by design PointNet does not capture local structures induced by the metric space points live in, limiting its ability to recognize fine-grained patterns and generalizability to complex scenes. In this work, we introduce a hierarchical neural network that applies PointNet recursively on a nested partitioning of the input point set. By exploiting metric space distances, our network is able to learn local features with increasing contextual scales. With further observation that point sets are usually sampled with varying densities, which results in greatly decreased performance for networks trained on uniform densities, we propose novel set learning layers to adaptively combine features from multiple scales. Experiments show that our network called PointNet++ is able to learn deep point set features efficiently and robustly. In particular, results significantly better than state-of-the-art have been obtained on challenging benchmarks of 3D point clouds. Unlike on images, semantic learning on 3D point clouds using a deep network is challenging due to the naturally unordered data structure. Among existing works, PointNet has achieved promising results by directly learning on point sets. However, it does not take full advantage of a point's local neighborhood that contains fine-grained structural information which turns out to be helpful towards better semantic learning. In this regard, we present two new operations to improve PointNet with a more efficient exploitation of local structures. The first one focuses on local 3D geometric structures. In analogy to a convolution kernel for images, we define a point-set kernel as a set of learnable 3D points that jointly respond to a set of neighboring data points according to their geometric affinities measured by kernel correlation, adapted from a similar technique for point cloud registration. The second one exploits local high-dimensional feature structures by recursive feature aggregation on a nearest-neighbor-graph computed from 3D positions. Experiments show that our network can efficiently capture local information and robustly achieve better performances on major datasets. Our code is available at http: www.merl.com research license#KCNet Unlike on images, semantic learning on 3D point clouds using a deep network is challenging due to the naturally unordered data structure. Among existing works, PointNet has achieved promising results by directly learning on point sets. However, it does not take full advantage of a point's local neighborhood that contains fine-grained structural information which turns out to be helpful towards better semantic learning. In this regard, we present two new operations to improve PointNet with a more efficient exploitation of local structures. The first one focuses on local 3D geometric structures. In analogy to a convolution kernel for images, we define a point-set kernel as a set of learnable 3D points that jointly respond to a set of neighboring data points according to their geometric affinities measured by kernel correlation, adapted from a similar technique for point cloud registration. The second one exploits local high-dimensional feature structures by recursive feature aggregation on a nearest-neighbor-graph computed from 3D positions. Experiments show that our network can efficiently capture local information and robustly achieve better performances on major datasets. Our code is available at this http URL Few prior works study deep learning on point sets. PointNet is a pioneer in this direction. However, by design PointNet does not capture local structures induced by the metric space points live in, limiting its ability to recognize fine-grained patterns and generalizability to complex scenes. In this work, we introduce a hierarchical neural network that applies PointNet recursively on a nested partitioning of the input point set. By exploiting metric space distances, our network is able to learn local features with increasing contextual scales. With further observation that point sets are usually sampled with varying densities, which results in greatly decreased performance for networks trained on uniform densities, we propose novel set learning layers to adaptively combine features from multiple scales. Experiments show that our network called PointNet++ is able to learn deep point set features efficiently and robustly. In particular, results significantly better than state-of-the-art have been obtained on challenging benchmarks of 3D point clouds.