@inproceedings{
gao2024multiscale,
title={Multi-Scale and Detail-Enhanced Segment Anything Model for Salient Object Detection},
author={Shixuan Gao and Pingping Zhang and Tianyu Yan and Huchuan Lu},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=G3MrTYNMtS}
}
Salient Object Detection (SOD) aims to identify and segment the most prominent objects in images. Existing methods on SOD utilize various Transformer-based models for feature extraction. However， due to the scale of training datasets and training methods, these Transformer-based models still lack performance and generalization in segmentation. Segment Anything Model (SAM) is trained on a large-scale segmentation dataset, which gives it strong generalization and segmentation capabilities. Nonetheless, SAM requires accurate prompts of target objects, which is unavailable in SOD. Additionally, SAM lacks the utilization of multi-scale and multi-layer information, as well as the incorporation of fine-grained details. In order to apply SAM to SOD, and address its shortcomings, we propose a Multi-scale and Detail-enhanced SAM (MDSAM). Specifically, we introduce a Lightweight Multi-scale Adapter (LMSA), which allows SAM to learn multi-scale information with few trainable parameters. Moreover, we propose a Multi-Layer Fusion Block (MLFB) to comprehensively utilize the multi-layer information from the SAM's encoder. Finally, we propose a Detail Enhancement Module (DEM) to incorporate SAM with fine-grained details. Experimental results demonstrate the superior performance of our model on multiple SOD datasets and its strong generalization to other segmentation tasks. The source code will be publicly available.
Multi-Scale and Detail-Enhanced Segment Anything Model for Salient Object Detection
Shixuan Gao Pingping Zhang Tianyu Yan Huchuan Lu