@inproceedings{
zhu2024calibration,
title={Calibration for Long-tailed Scene Graph Generation},
author={XuHan Zhu and Yifei Xing and Ruiping Wang and Yaowei Wang and Xiangyuan Lan},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=jGNDRM2vul}
}
Miscalibrated models tend to be unreliable and insecure for downstream applications. In this work, we attempt to highlight and remedy miscalibration in current scene graph generation (SGG) models, which has been overlooked by previous works. We discover that obtaining well-calibrated models for SGG is more challenging than conventional calibration settings, as long-tailed SGG training data exacerbates miscalibration with overconfidence in head classes and underconfidence in tail classes. We further analyze which components are explicitly impacted by the long-tailed data during optimization, thereby exacerbating miscalibration and unbalanced learning, including: \textbf{biased parameters}, \textbf{deviated boundaries}, and \textbf{distorted target distribution}. To address the above issues, we propose the \textbf{C}ompositional \textbf{O}ptimization \textbf{C}alibration (\textbf{COC}) method, comprising three modules: i. A parameter calibration module that utilizes a hyperspherical classifier to eliminate the bias introduced by biased parameters. ii. A boundary calibration module that disperses features of majority classes to consolidate the decision boundaries of minority classes and mitigate deviated boundaries. iii. A target distribution calibration module that addresses distorted target distribution, leverages within-triplet prior to guide confidence-aware and label-aware target calibration, and applies curriculum regulation to constrain learning focus from easy to hard classes. Extensive evaluation on popular benchmarks demonstrates the effectiveness of our proposed method in improving model calibration and resolving unbalanced learning for long-tailed SGG. Finally, our proposed method performs best on model calibration compared to different types of calibration methods and achieves state-of-the-art trade-off performance on balanced learning for SGG. The source codes and models will be available upon acceptance.
Calibration for Long-tailed Scene Graph Generation
XuHan Zhu Yifei Xing Ruiping Wang Yaowei Wang Xiangyuan Lan