@inproceedings{
he2024textprompt,
title={Text-prompt Camouflaged Instance Segmentation with Graduated Camouflage Learning},
author={zhentao he and Changqun Xia and Shengye Qiao and Jia Li},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=M33uCfAfgG}
}
Camouflaged instance segmentation (CIS) aims to seamlessly detect and segment objects blending with their surroundings. While existing CIS methods rely heavily on fully-supervised training with massive precisely annotated data, consuming considerable annotation efforts yet struggling to segment highly camouflaged objects accurately. 
Despite their visual similarity to the background, camouflaged objects differ semantically. Since text associated with images offers explicit semantic cues to underscore this difference, in this paper we propose a novel approach: the first \textbf{T}ext-\textbf{P}rompt based weakly-supervised camouflaged instance segmentation method named TPNet, leveraging semantic distinctions for effective segmentation. 
Specifically, TPNet operates in two stages: initiating with the generation of pseudo masks followed by a self-training process. 
In the pseudo mask generation stage, we innovatively align text prompts with images using a pre-training language-image model to obtain region proposals containing camouflaged instances and specific text prompt. Additionally, a Semantic-Spatial Iterative Fusion module is ingeniously designed to assimilate spatial information with semantic insights, iteratively refining pseudo mask.
In the following stage, we employ Graduated Camouflage Learning, a straightforward self-training optimization strategy that evaluates camouflage levels to sequence training from simple to complex images, facilitating for an effective learning gradient.
Through the collaboration of the dual phases, our method offers a comprehensive experiment on two common benchmark and demonstrates a significant advancement, delivering a novel solution that bridges the gap between weak-supervised and high camouflaged instance segmentation.
Text-prompt Camouflaged Instance Segmentation with Graduated Camouflage Learning
zhentao he Changqun Xia Shengye Qiao Jia Li