@inproceedings{
zhu2024adaptive,
title={Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Compare},
author={Hanwei Zhu and Haoning Wu and Yixuan Li and Zicheng Zhang and Baoliang Chen and Lingyu Zhu and Yuming Fang and Guangtao Zhai and Weisi Lin and Shiqi Wang},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=mHtOyh5taj}
}
While recent advancements in large multimodal models (LMMs) have significantly improved their abilities in image quality assessment (IQA) relying on absolute quality rating, how to transfer reliable relative quality comparison outputs to continuous perceptual quality scores remains largely unexplored. To address this gap, we introduce an all-around LMM-based NR-IQA model, which is capable of producing qualitatively comparative responses and effectively translating these discrete comparison outcomes into a continuous quality score. Specifically, during training, we present to generate scaled-up comparative instructions by comparing images from the same IQA dataset, allowing for more flexible integration of diverse IQA datasets. Utilizing the established large-scale training corpus, we develop a human-like visual quality comparator. During inference, moving beyond binary choices, we propose a soft comparison method that calculates the likelihood of the test image being preferred over multiple predefined anchor images. The quality score is further optimized by maximum a posteriori estimation with the resulting probability matrix. Extensive experiments on nine IQA datasets validate that the Compare2Score effectively bridges text-defined comparative levels during training with converted single image quality scores for inference, surpassing state-of-the-art IQA models across diverse scenarios. Moreover, we verify that the probability-matrix-based inference conversion not only improves the rating accuracy of Compare2Score but also zero-shot general-purpose LMMs, suggesting its intrinsic effectiveness.
Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Compare
Hanwei Zhu Haoning Wu Yixuan Li Zicheng Zhang Baoliang Chen Lingyu Zhu Yuming Fang Guangtao Zhai Weisi Lin Shiqi Wang