For these reasons, as well as the availability of large-scale legal corpora from the web, we chose to focus on _continued pretraining_. We meticulously curate a high-quality dataset sourced from diverse legal content repositories. After rigorous filtering Penedo et al. (2023) and deduplication Mou et al. (2023); Kocetkov et al. (2023), we end up with a corpus of \(30\) billion tokens, which serves as a robust foundation for continued pretraining.