In this paper, we introduce SaulLM-7B, an open-source decoder model delivering state-of-the-art performance, compared to 7B models, within the legal domain. Our approach entails fine-tuning legal data alongside instruction fine-tuning on synthetic datasets. Additionally, we contribute by providing a cleaned version of LegalBench and introducing a new set of documents for perplexity measurement. We hope that our model, which is released under the MIT license, will contribute to the open-source ecosystem and the community.