However, despite those breakthroughs in medical segmentation, current practices of medical segmentation mainly focus on the binary segmentation of lesions, i.e. distinguishing lesions (foreground) and everything else (background). Although binary segmentation does help improve the clinical decisions of physicians, the information provided by the binary segmentation is still limited. In clinical scenarios, physicians usually need to distinguish different types of lesions, i.e. multi-lesion segmentation. Compared with binary segmentation, such scenarios are much more difficult due to the interclass similarity of the tissues, as different kinds of lesions can resemble in texture, size and shape. Cascaded networks with a coarse-to-fine strategy have been extensively used in such scenarios, such as the segmentation of liver and lesions, and brain tumor segmentation [1]} [2]} [3]} [4]}. Such networks usually consist of two independent networks, where the first network performs the coarse segmentation and the second network refines the segmentation based on the ROIs segmented from the first network. However, although cascaded networks have been widely used in multi-lesion segmentation in medical images, the cascade strategy has its own drawbacks. Since the cascaded network consists of two independent networks, the parameter amount and video memory occupation are typically twice those of a single network, which could be demanding in hardware and limits its potential in clinical usage. More importantly, as the two networks in a cascaded network are usually independent, the training process of the cascaded network is sometimes more difficult than a single network, which could lead to underfitting.
In this paper, we present a novel network structure named Prior Attention Network (PANet) to perform multi-lesion segmentation in medical images. The proposed network consists of an encoder for feature extraction, and two decoders to generate lesion region attention and final predictions separately. The network is combined together with attention mechanism. To reduce the parameter size and hardware occupation, we use the deep, semantic-rich features from the network encoder to generate the spatial attention of lesion regions. The feature representations generated by the encoder are then refined by the spatial attention and send to the decoder for final multi-class predictions. To improve segmentation performance and accelerate the convergence, we also introduce intermediate supervision and deep supervision to the network structure. With these improvements, the proposed network achieves competitive results with significantly lower parameter size and computational cost compared with traditional cascaded networks.
