@inproceedings{
nguyen2023on,
title={On the Out of Distribution Robustness of Foundation Models in Medical Image Segmentation},
author={Duy Minh Ho Nguyen and Tan Ngoc Pham and Nghiem Tuong Diep and Nghi Quoc Phan and Quang Pham and Vinh Tong and Binh T. Nguyen and Ngan Hoang Le and Nhat Ho and Pengtao Xie and Daniel Sonntag and Mathias Niepert},
booktitle={R0-FoMo:Robustness of Few-shot and Zero-shot Learning in Large Foundation Models},
year={2023},
url={https://openreview.net/forum?id=CPpkklQWQW}
}
Constructing a robust model that can effectively generalize to test samples under
distribution shifts remains a significant challenge in the field of medical imaging. The foundational models for vision and language, pre-trained on extensive sets of natural image and text data, have emerged as a promising approach. It showcases impressive learning abilities across different tasks with the need for only a limited amount of annotated samples. While numerous techniques have
focused on developing better fine-tuning strategies to adapt these models for specific domains, we instead examine their robustness to domain shifts in the medical image segmentation task. To this end, we compare the generalization performance to unseen domains of various pre-trained models after being fine-tuned on the same in-distribution dataset and show that foundation-based models enjoy better robustness than other architectures. From here, we further developed a new Bayesian uncertainty estimation for frozen models and used them as an indicator to characterize the model’s performance on out-of-distribution (OOD) data, proving particularly beneficial for real-world applications. Our experiments not only reveal the limitations of current indicators like accuracy on the line or agreement on the line commonly used in natural image applications but also emphasize the promise of the introduced Bayesian uncertainty. Specifically, lower uncertainty predictions.
usually tend to higher out-of-distribution (OOD) performance.
On the Out of Distribution Robustness of Foundation Models in Medical Image Segmentation
Duy Minh Ho Nguyen Tan Ngoc Pham Nghiem Tuong Diep Nghi Quoc Phan Quang Pham Vinh Tong Binh T. Nguyen Ngan Hoang Le Nhat Ho Pengtao Xie Daniel Sonntag Mathias Niepert