@inproceedings{
wu2024crossmodel,
title={Cross-model Control: Improving Multiple Large Language Models in One-time Training},
author={Jiayi Wu and Hao Sun and Hengyi Cai and Lixin Su and Shuaiqiang Wang and Dawei Yin and Xiang Li and Ming Gao},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=YPqHSTSoFs}
}
The number of large language models (LLMs) with varying parameter scales and vocabularies is increasing. While they deliver powerful performance, they also face a set of common optimization needs to meet specific requirements or standards, such as instruction following or avoiding the output of sensitive information from the real world. However, how to reuse the fine-tuning outcomes of one model to other models to reduce training costs remains a challenge. To bridge this gap, we introduce Cross-model Control (CMC), a method that improves multiple LLMs in one-time training with a portable tiny language model. Specifically, we have observed that the logit shift before and after fine-tuning is remarkably similar across different models. Based on this insight, we incorporate a tiny language model with a minimal number of parameters. By training alongside a frozen template LLM, the tiny model gains the capability to alter the logits output by the LLMs. To make this tiny language model applicable to models with different vocabularies, we propose a novel token mapping strategy named PM-MinED. We have conducted extensive experiments on instruction tuning and unlearning tasks, demonstrating the effectiveness of CMC. Our code is available at https://github.com/wujwyi/CMC
Cross-model Control: Improving Multiple Large Language Models in One-time Training
Jiayi Wu Hao Sun Hengyi Cai Lixin Su Shuaiqiang Wang Dawei Yin Xiang Li Ming Gao