@inproceedings{
feng2024cpprompt,
title={{CP}-Prompt: Composition-Based Cross-modal Prompting for Domain-Incremental Continual Learning},
author={Yu Feng and Zhen Tian and Yifan Zhu and Zongfu Han and Haoran Luo and Guangwei Zhang and Meina Song},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=sZn9Hq0mIQ}
}
The key challenge of cross-modal domain-incremental learning (DIL) is to enable the learning model to continuously learn from novel data with different feature distributions under the same task without forgetting old ones. However, existing top-performing methods still cause high forgetting rates, by lacking intra-domain knowledge extraction and inter-domain common prompting strategy. In this paper, we propose a simple yet effective framework, CP-Prompt, by training limited parameters to instruct a pre-trained model to learn new domains and avoid forgetting existing feature distributions. CP-Prompt captures intra-domain knowledge by compositionally inserting personalized prompts on multi-head self-attention layers and then learns the inter-domain knowledge with a common prompting strategy. 
CP-Prompt shows superiority compared with state-of-the-art baselines among three widely evaluated DIL tasks. 
The source code is available at https://anonymous.4open.science/r/CP_Prompt-C126.
CP-Prompt: Composition-Based Cross-modal Prompting for Domain-Incremental Continual Learning
Yu Feng Zhen Tian Yifan Zhu Zongfu Han Haoran Luo Guangwei Zhang Meina Song