@inproceedings{
hans2024be,
title={Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative {LLM}s},
author={Abhimanyu Hans and John Kirchenbauer and Yuxin Wen and Neel Jain and Hamid Kazemi and Prajwal Singhania and Siddharth Singh and Gowthami Somepalli and Jonas Geiping and Abhinav Bhatele and Tom Goldstein},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=DylSyAfmWs}
}
Large language models can memorize and repeat their training data, causing privacy and copyright risks. To mitigate memorization, we introduce a subtle modification to the next-token training objective that we call the goldfish loss. During training, a randomly sampled subsets of tokens are excluded from the loss computation. These dropped tokens are not memorized by the model, which prevents verbatim reproduction of a complete chain of tokens from the training set. We run extensive experiments training billion-scale LLaMA-2 models, both pre-trained and trained from scratch, and demonstrate significant reductions in extractable memorization with little to no impact on downstream benchmarks.

_Code and checkpoints: https://github.com/ahans30/goldfish-loss_
Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs
Abhimanyu Hans John Kirchenbauer Yuxin Wen Neel Jain Hamid Kazemi Prajwal Singhania Siddharth Singh Gowthami Somepalli Jonas Geiping Abhinav Bhatele Tom Goldstein