@inproceedings{
yan2024categoryprompt,
title={Category-Prompt Refined Feature Learning for Long-Tailed Multi-Label Image Classification},
author={Jiexuan Yan and Sheng Huang and Nankun Mu and Luwen Huangfu and Bo Liu},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=ga3z0S7Ahg}
}
Real-world data consistently exhibits a long-tailed distribution, often spanning multiple categories. This complexity underscores the challenge of content comprehension, particularly in scenarios requiring long-tailed multi-label image classification (LTMLC). In such contexts, imbalanced data distribution and multi-object recognition pose significant hurdles. To address this issue, we propose a novel and effective approach for LTMLC, termed Category-Prompt Refined Feature Learning (CPRFL), utilizing semantic correlations between different categories and decoupling category-specific visual representations for each category. Specifically, CPRFL initializes category-prompts from the pretrained CLIP's embeddings and decouples category-specific visual representations through interaction with visual features, thereby facilitating the establishment of semantic correlations between the head and tail classes. To mitigate the visual-semantic domain bias, we design a progressive Dual-Path Back-Propagation mechanism to refine the prompts by progressively incorporating context-related visual information into prompts. Simultaneously, the refinement process facilitates the progressive purification of the category-specific visual representations under the guidance of the refined prompts. Furthermore, taking into account the negative-positive sample imbalance, we adopt the Asymmetric Loss as our optimization objective to suppress negative samples across all classes and potentially enhance the head-to-tail recognition performance. We validate the effectiveness of our method on two LTMLC benchmarks and extensive experiments demonstrate the superiority of our work over baselines.
Category-Prompt Refined Feature Learning for Long-Tailed Multi-Label Image Classification
Jiexuan Yan Sheng Huang Nankun Mu Luwen Huangfu Bo Liu