Our findings place domain continual pre-training as a strong alternative to building domain-specific LLMs from scratch. By being smarter about data selection for continual pre-training, we can surpass vanilla continual pre-training at a fraction of the cost. Overall, our work paves the way for developing domain-specific LLMs at a reduced cost, with implications for a wide range of applications.