confidential data. Domain-adaptive continual pre-training (DACP) is a straightforward alternative to building from scratch; we continually pre-train a general-purpose LLM on a large scale corpus of domain-specific unlabeled data. Domain-adaptive continual pre-training has shown the ability to adapt the language models to better fit the in-domain distribution [8; 12; 32; 21]. They also enable large language models to acquire new knowledge as new data appears [11; 10], instead of training the model from scratch. We use DACP in our experiments to benchmark its benefits.