@inproceedings{
deng2024simclip,
title={Sim{CLIP}: Refining Image-Text Alignment with Simple Prompts for Zero-/Few-shot Anomaly Detection},
author={ChengHao Deng and haote xu and Xiaolu Chen and Haodi Xu and Xiaotong Tu and Xinghao Ding and Yue Huang},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=kiH6PqRhwE}
}
Recently, large pre-trained vision-language models, such as CLIP, have demonstrated significant potential in zero-/few-shot anomaly detection tasks. However, existing methods not only rely on expert knowledge to manually craft extensive text prompts but also suffer from a misalignment of high-level language features with fine-level vision features in anomaly segmentation tasks. In this paper, we propose a method, named SimCLIP, which focuses on refining the aforementioned misalignment problem through bidirectional adaptation of both Multi-Hierarchy Vision Adapter (MHVA) and Implicit Prompt Tuning (IPT). In this way, our approach requires only a simple binary prompt to accomplish anomaly classification and segmentation tasks in zero-shot scenarios efficiently. Furthermore, we introduce its few-shot extension, SimCLIP+, integrating the relational information among vision embedding and skillfully merging the cross-modal synergy information between vision and language to address AD tasks. Extensive experiments on two challenging datasets prove the more remarkable generalization capacity of our method compared to the current state-of-the-art.
SimCLIP: Refining Image-Text Alignment with Simple Prompts for Zero-/Few-shot Anomaly Detection
ChengHao Deng haote xu Xiaolu Chen Haodi Xu Xiaotong Tu Xinghao Ding Yue Huang