Object instance segmentation is one of the most fundamental but challenging tasks in computer vision, and it requires the pixel-level image understanding. Most existing approaches address this problem by adding a mask prediction branch to a two-stage object detector with the Region Proposal Network (RPN). Although producing good segmentation results, the efficiency of these two-stage approaches is far from satisfactory, restricting their applicability in practice. In this paper, we propose a one-stage framework, SPRNet, which performs efficient instance segmentation by introducing a single pixel reconstruction (SPR) branch to off-the-shelf one-stage detectors. The added SPR branch reconstructs the pixel-level mask from every single pixel in the convolution feature map directly. Using the same ResNet-50 backbone, SPRNet achieves comparable mask AP to Mask R-CNN at a higher inference speed, and gains all-round improvements on box AP at every scale comparing with RetinaNet.
In both the detection and segmentation tasks, the problem of detecting small objects is an open problem. On the bottom layers of a deep networks, objects have rich spatial information but lacking of clear semantic; while on the top layers the situation turn around. To tackle this problem, Feature Pyramid Network (FPN) was proposed to aggregate multi-level features in a pyramidal hierarchy. Beyond the traditional pathway by the deep networks, (FPN) additionally established a paths to deliver the rich semantic information of top layers to the bottom layers @cite_14 . In their implementation, each top layer feature is 2 @math up-sampled and then merged with the bottom level feature using element-wise summation. By doing so, the semantic information of top layers can be passed to the bottom layers successfully. However, their inaccurate spatial information is also passed to the bottom layers at the same time, which the effectiveness to some extent.
@cite_14
2949533892
Feature pyramids are a basic component in recognition systems for detecting objects at different scales. But recent deep learning object detectors have avoided pyramid representations, in part because they are compute and memory intensive. In this paper, we exploit the inherent multi-scale, pyramidal hierarchy of deep convolutional networks to construct feature pyramids with marginal extra cost. A top-down architecture with lateral connections is developed for building high-level semantic feature maps at all scales. This architecture, called a Feature Pyramid Network (FPN), shows significant improvement as a generic feature extractor in several applications. Using FPN in a basic Faster R-CNN system, our method achieves state-of-the-art single-model results on the COCO detection benchmark without bells and whistles, surpassing all existing single-model entries including those from the COCO 2016 challenge winners. In addition, our method can run at 5 FPS on a GPU and thus is a practical and accurate solution to multi-scale object detection. Code will be made publicly available.