@inproceedings{
chen2024fodfom,
title={FodFoM: Fake Outlier Data by Foundation Models Creates Stronger Visual Out-of-Distribution Detector},
author={Jiankang Chen and Ling Deng and Zhiyong Gan and Wei-Shi Zheng and Ruixuan Wang},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=iWSVl6mLbW}
}
Out-of-Distribution (OOD) detection is crucial when deploying machine learning models in open-world applications. The core
challenge in OOD detection is mitigating the model’s overconfidence on OOD data. While recent methods using auxiliary outlier
datasets or synthesizing outlier features have shown promising OOD detection performance, they are limited due to costly data col-
lection or simplified assumptions. In this paper, we propose a novel OOD detection framework FodFoM that innovatively combines
multiple foundation models to generate two types of challenging fake outlier images for classifier training. The first type is based
on BLIP-2’s image captioning capability, CLIP’s vision-language knowledge, and Stable Diffusion’s image generation ability. Jointly
utilizing these foundation models constructs fake outlier images which are semantically similar to but different from in-distribution
(ID) images. For the second type, GroundingDINO’s object detection ability is utilized to help construct pure background images by blur-
ring foreground ID objects in ID images. The proposed framework can be flexibly combined with multiple existing OOD detection
methods. Extensive empirical evaluations show that image classifiers with the help of constructed fake images can more accurately
differentiate real OOD image from ID ones. New state-of-the-art OOD detection performance is achieved on multiple benchmarks.
The source code will be publicly released.
FodFoM: Fake Outlier Data by Foundation Models Creates Stronger Visual Out-of-Distribution Detector
Jiankang Chen Ling Deng Zhiyong Gan Wei-Shi Zheng Ruixuan Wang