In contrast to the existing approaches, MT-TransUNet offers the following three unique advantages.
(1) Aggregating long-range dependencies in the image. Unlike conventional CNN architectures, the self-attention layers in the Transformer architecture are capable of capturing long-range dependencies [1]}, [2]}, which shed new light on identifying larger skin lesions from dermoscopic images.
(2) Exploiting both pixel- and image-level annotation. Although Y-Net [3]} attempted to combine segmentation and classification back in 2018, there was little performance gain owing to the asymmetrical learning objectives of the two tasks. We have overcome this barrier by regulating the internal consistency between the class attention map and lesion segmentation map.
(3) Demonstrating superior computational efficiency. The previous state of the art, MB-DCNN [4]}, was extremely time-consuming and resource-intensive for training and testing, as the three modules in their architecture must be trained individually. Besides outperforming MB-DCNN by a large margin, our MT-TransUNet also presents a remarkable improvement in efficiency thanks to the design of a shared encoder.
