@inproceedings{
ru2024parameterefficient,
title={Parameter-Efficient Complementary Expert Learning for Long-Tailed Visual Recognition},
author={Lixiang Ru and Xin Guo and Lei Yu and Yingying Zhang and Jiangwei Lao and Jian Wang and Jingdong Chen and Yansheng Li and Ming Yang},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=OiPyYoPhAd}
}
Long-tailed recognition (LTR) aims to learn balanced models from extremely unbalanced training data. Fine-tuning pretrained foundation models has recently emerged as a promising research direction for LTR. However, we observe that the fine-tuning process tends to degrade the intrinsic representation capability of pretrained models and lead to model bias towards certain classes, thereby hindering the overall recognition performance. To unleash the intrinsic representation capability of pretrained foundation models, in this work, we propose a new Parameter-Efficient Complementary Expert Learning (PECEL) for LTR. Specifically, PECEL consists of multiple experts, where individual experts are trained via Parameter-Efficient Fine-Tuning (PEFT) and encouraged to learn different expertise on complementary sub-categories via a new sample-aware logit adjustment loss. By aggregating the predictions of different experts, PECEL effectively achieves a balanced performance on long-tailed classes. Nevertheless, learning multiple experts generally introduces extra trainable parameters. To ensure parameter efficiency, we further propose a parameter sharing strategy which decomposes and shares the parameters in each expert. Extensive experiments on 4 LTR benchmarks show that the proposed PECEL can effectively learn multiple complementary experts without increasing the trainable parameters and achieve new state-of-the-art performance.
Parameter-Efficient Complementary Expert Learning for Long-Tailed Visual Recognition
Lixiang Ru Xin Guo Lei Yu Yingying Zhang Jiangwei Lao Jian Wang Jingdong Chen Yansheng Li Ming Yang