In this paper, we focus on the voxel representation of large-scale point clouds. The voxel representation is widely used in 3D object detection [1]}, [2]}, [3]}. The 3D Sparse Convolutions [2]} with positional encoding can focus on voxels with point clouds thus reducing computation cost. The common voxel representation learning operators include mean operator, random sampling, and MLP operator [5]}. Reconstructing the voxel features as a regression task is very difficult as the pre-training network needs to learn the distributions of each point. However, whether the voxel contains point clouds is more important to 3D object detectors. Thus, unlike the pixel reconstruction task in MAE for 2D images, we design the masked voxel classification task for large-scale point clouds, called Voxel-MAE, as illustrated in Fig. REF . That is, we first randomly mask some voxels, then put the masked voxels into the 3D encoder built by 3D Sparse Convolutions and the output of the 3D decoder built by 3D Deconvolutions is the probability of containing point clouds for each voxel, at last, we calculate the binary classification loss for the pre-training network. This masked voxel classification task pushes the encoder network to be voxel-aware to the whole shape of objects in point clouds, thus learning representative features for 3D object detection.
