@inproceedings{
xie2024qptv,
title={{QPT}-V2: Masked Image Modeling Advances Visual Scoring},
author={Qizhi Xie and Kun Yuan and Yunpeng Qu and Mingda Wu and Ming Sun and Chao Zhou and Jihong Zhu},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=srGZPS1rsN}
}
Quality assessment and aesthetics assessment aim to evaluate the perceived quality and aesthetics of visual content. Current learning-based methods suffer greatly from the scarcity of labeled data and usually perform sub-optimally in terms of generalization. Although masked image modeling (MIM) has achieved noteworthy advancements across various high-level tasks (i.e., classification, detection etc.). In this work, we take on a novel perspective to investigate its capabilities in terms of quality- and aesthetics-awareness. To this end, we propose Quality- and aesthectics-aware PreTraining (QPT V2), the first pretraining framework based on MIM that offers a unified solution for quality and asthectics assessment. Specifically, QPT V2 incporporates following key designs: To perceive the high-level semantics and fine-grained details, pretraining data is curated. To comprehensively encompass quality- and aesthetics-related factors, degradation is introduced. To capture multi-scale quality and aesthetics information, model structure is modified. Extensive experimental results on 11 downstream benchmarks clearly show the superior performance of QPT V2 in comparison with current state-of-the-art approaches and other pretraining paradigms.
QPT-V2: Masked Image Modeling Advances Visual Scoring
Qizhi Xie Kun Yuan Yunpeng Qu Mingda Wu Ming Sun Chao Zhou Jihong Zhu