Pixel-level semantic segmentation is a challenging task with a huge amount of computation, especially if the input sizes are large. In the segmentation network, apart from the pyramid backbone network, an extra decoder network is often employed to recover the spatial detail information. In this paper, we put forward a method for single-shot segmentation in a feature residual pyramid network (RPNet), which learns the coarse results and residuals of segmentations by decomposing the label at different levels of residual blocks. Specifically speaking, we use the residual features to learn the edges and details, and we also use the top-level feature to learn the coarse segmentation result. At the testing phase, the predicted residuals are used to enhance the details of the coarse segmentation result. Residual learning blocks split the network into several shallow sub-networks by level-wise training, which facilitates the gradient propagation in the RPNet. We then evaluate the proposed method and compare it with the recent state-of-the-art methods on CamVid and Cityscapes datasets. The proposed single-shot segmentation based on the RPNet achieves impressive results with high efficiency on the pixel-level segmentation task.
Real-time segmentation models prefer thin networks with fewer filters so that computing cost can be reduced such as ENet @cite_3 . But simply reducing computation will lead to degradation in performance.
@cite_3
2793866399
Real-time semantic segmentation is of significant importance for mobile and robotics related applications. We propose a computationally efficient segmentation network which we term as ShuffleSeg. The proposed architecture is based on grouped convolution and channel shuffling in its encoder for improving the performance. An ablation study of different decoding methods is compared including Skip architecture, UNet, and Dilation Frontend. Interesting insights on the speed and accuracy tradeoff is discussed. It is shown that skip architecture in the decoding method provides the best compromise for the goal of real-time performance, while it provides adequate accuracy by utilizing higher resolution feature maps for a more accurate segmentation. ShuffleSeg is evaluated on CityScapes and compared against the state of the art real-time segmentation networks. It achieves 2x GFLOPs reduction, while it provides on par mean intersection over union of 58.3 on CityScapes test set. ShuffleSeg runs at 15.7 frames per second on NVIDIA Jetson TX2, which makes it of great potential for real-time applications.