Domain specific large language models.While the majority of released LLMs are general-purpose models, domain-specific LLMs have emerged as valuable counterparts. Google's MedPaLM and MedPaLM-2, trained on a medical domain corpus, achieved state-of-the-art results on medical benchmarks [23; 24]. Bloomberg developed the first financial LLM from scratch by training on a financial corpus [31] while Galactica was developed for scientific domains [26]. Continual pre-training presents an alternative approach to building domain-specific LLMs from scratch.