Meltemi: The first open Large Language Model for Greek
IAMJB
Abstract
We describe the development and capabilities of Meltemi 7B, the first open Large Language Model for the Greek language. Meltemi 7B has 7 billion parameters and is trained on a 40 billion token Greek corpus. For the development of Meltemi 7B, we adapt Mistral, by continuous pretraining on the Greek Corpus. Meltemi 7B contains up-to-date information up to September 2023. Furthermore, we have translated and curated a Greek instruction corpus, which has been used for the instruction-tuning of a chat model, named Meltemi 7B Instruct. Special care has been given to the alignment and the removal of toxic content for the Meltemi 7B Instruct. The developed models are evaluated on a broad set of collected evaluation corpora, and examples of prompts and responses are presented. Both Meltemi 7B and Meltemi 7B Instruct are available at https://huggingface.co/ilsp under the Apache 2.0 license.
Community
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Adapting LLMs to Hebrew: Unveiling DictaLM 2.0 with Enhanced Vocabulary and Instruction Capabilities (2024)
- Open Generative Large Language Models for Galician (2024)
- "Vorbeşti Româneşte?" A Recipe to Train Powerful Romanian LLMs with English Instructions (2024)
- FuLG: 150B Romanian Corpus for Language Model Pretraining (2024)
- Bilingual Adaptation of Monolingual Foundation Models (2024)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
What about Greek BERT: https://huggingface.co/nlpaueb/bert-base-greek-uncased-v1 ? it's not a LLM? 😁
from @nlpaueb
Hello julien,
currently even low-billion parameter count LMs are sometimes referred to as SLMs (even 7B ones such as Meltemi, to be fair), so the usage of the term LLM is inherently subjective. Parameter count aside, we use the term LLM to describe models that are designed for generative tasks, and while BERT models serve as foundational models for many NLP tasks, they aren't necessarily designed with generative tasks in mind
Models citing this paper 10
Browse 10 models citing this paperDatasets citing this paper 0
No dataset linking this paper