Point clouds are a flexible and ubiquitous way to represent 3D objects with arbitrary resolution and precision. Previous work has shown that adapting encoder networks to match the semantics of their input point clouds can significantly improve their effectiveness over naive feedforward alternatives. However, the vast majority of work on point-cloud decoders are still based on fully-connected networks that map shape representations to a fixed number of output points. In this work, we investigate decoder architectures that more closely match the semantics of variable sized point clouds. Specifically, we study sample-based point-cloud decoders that map a shape representation to a point feature distribution, allowing an arbitrary number of sampled features to be transformed into individual output points. We develop three sample-based decoder architectures and compare their performance to each other and show their improved effectiveness over feedforward architectures. In addition, we investigate the learned distributions to gain insight into the output transformation. Our work is available as an extensible software platform to reproduce these results and serve as a baseline for future work. Point clouds are an important data type for deep learning algorithms to support. They are commonly used to represent point samples of some underlying object. More generally, the points may be extended beyond 3D space to capture additional information about multi-sets of individual objects from some class. The key distinction between point clouds and the more typical tensor data types is that the information content is invariant to the ordering of points. This implies that the spatial relationships among points is not explicitly captured via the indexing structure of inputs and outputs. Thus, standard convolutional architectures, which leverage such indexing structure to support spatial generalization, are not directly applicable. A common approach to processing point clouds with deep networks is voxelization, where point clouds are represented by one or more occupancy-grid tensors (Zhou & Tuzel (2018) , Wu et al. (2018) ). The grids encode the spatial dimensions of the points in the tensor indexing structure, which allows for the direct application of convolutional architectures. This voxelization approach, however, is not appropriate in many use cases. In particular, the size of the voxelized representation depends on the spatial extent of the point cloud relative to the spatial resolution needed to make the necessary spatial distinctions (such as distinguishing between different objects in LIDAR data). In many cases, the required resolution will be unknown or result in enormous tensors, which can go beyond the practical space and time constraints of an application. This motivates the goal of developing architectures that support processing point cloud data directly, so that processing scales with the number of points rather than the required size of an occupancy grid. One naive approach, which scales linearly in the size of the point cloud, is to 'flatten' the point cloud into an arbitrarily ordered list. The list can then be directly processed by standard convolutional or fully-connected (MLP) architectures directly. This approach, however, has at least two problems. First, the indexing order in the list carries no meaningful information, while the networks do not encode this as a prior. Thus, the networks must learn to generalize in a way that is invariant to ordering, which can be data inefficient. Second, in some applications, it is useful for point clouds to consist of varying numbers of points, while still representing the same underlying objects. However, the number of points that can be consumed by the naive feedforward architecture is fixed. PointNet (Qi et al., 2017) and Deepsets Zaheer et al. (2017) exhibit better performance over the MLP baseline with a smaller network by independently transforming each point into a high-dimensional representation with a single shared MLP that is identically applied to each individual point. This set of derived point features is then mapped to a single, fixed-sized dense shape representation using a symmetric reduction function. As such the architectures naturally scale to any number of input points and order invariance is built in as an architectural bias. As a result, these architectures have been shown to yield significant advantages in applications in which point clouds are used as input, such as shape classification. The success of PointNet and DeepSet style architectures in this domain shows that designing a network architecture to match the semantics of a point cloud results in a more efficient, and better performing network. Since point clouds are such a useful object representation, it's natural to ask how we should design networks to decode point clouds from some provided shape representation. This would allow for the construction of point cloud auto-encoders, which could serve a number of applications, such as anomaly detection and noise smoothing. Surprisingly, the dominant approach to designing such a differentiable point cloud decoder is to feed the dense representation of the desired object through a single feedforward MLP whose result is then reshaped into the appropriate size for the desired point cloud. This approach has similar issues as the flat MLP approach to encoding point clouds; the decoder can only produce a fixed-sized point cloud while point clouds are capable of representing objects at low or high levels of detail; the decoder only learns a single deterministic mapping from a shape representation to a point cloud while we know that point clouds are inherently random samples of the underlying object. The primary goal and contribution of this paper is to study how to apply the same lessons learned from the PointNet encoder's semantic congruence with point clouds to a point cloud decoder design. As such, we build on PointNet's principles to present the 'NoiseLearn' algorithm-a novel, simple, and effective point cloud decoding approach. The simplicity of the decoding architectures and the increase in performance are strong indicators that sample-based decoders should be considered as a default in future studies and systems. In addition, we investigate the operation of the decoders to gain insight into how the output point clouds are generated from a latent shape representation. In this work, we evaluated and compared several realizations of a sample-based point cloud decoder architecture. We show that these sampling approaches are competitive with or outperform the MLP approach while using fewer parameters and providing better functionality. These advantages over the baseline suggest that sample based point cloud decoders should be the default approach when a network needs to produce independent point samples of some underlying function or object. To further this this area of research, we provide a complete open-source implementation of our tools used to train and evaluate these networks.
We present and evaluate sampling-based point cloud decoders that outperform the baseline MLP approach by better matching the semantics of point clouds.
{"src": "scitldr-AIC_train", "paper_id": "SklVI1HKvH"}