Our proposed network architecture is built upon the SalsaNext [1]}, a single encoder-and-decoder network for LiDAR range image-based semantic segmentation.
To make it suitable for MOS, we extend and modify it into a dual-branch and dual-head network, consisting of a range image branch (Enc-A) to encode the appearance feature, a residual image branch (Enc-M) to encode the temporal motion information, an image head (ImageHead) with skip connections to decode the features from both Enc-A and Enc-M, and a point head (PointHead) to further refine the segmentation results.
Specifically, in the feature encoding stage, we first use the meta-kernel operator to better capture the 3D spatial information, and then use a motion-guided attention module to more effectively fuse the motion information extracted from residual images. In the final stage of decoding, besides the loss on the range image dealt with the ImageHead, we also back-project the 2D features to 3D point clouds, and use a lightweight sparse convolution module (PointHead) to refine the segmentation results.
