Update README.md
Browse files
README.md
CHANGED
@@ -16,7 +16,7 @@ inference:
|
|
16 |
|
17 |
# Model Card for DictaLM-2.0-AWQ
|
18 |
|
19 |
-
The DictaLM-2.0 Large Language Model (LLM) is a pretrained generative text model with 7 billion parameters
|
20 |
|
21 |
For full details of this model please read our [release blog post](https://example.com).
|
22 |
|
@@ -56,7 +56,7 @@ print(model(prompt.strip(), do_sample=False, max_new_tokens=4, stop_sequence='\n
|
|
56 |
## Model Architecture
|
57 |
|
58 |
DictaLM-2.0 is based on the [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) model with the following changes:
|
59 |
-
- An extended tokenizer with tokens for Hebrew, increasing the compression
|
60 |
- Continued pretraining on over 190B tokens of naturally occuring text, 50% Hebrew and 50% English.
|
61 |
|
62 |
## Notice
|
|
|
16 |
|
17 |
# Model Card for DictaLM-2.0-AWQ
|
18 |
|
19 |
+
The DictaLM-2.0 Large Language Model (LLM) is a pretrained generative text model with 7 billion parameters trained to specialize in Hebrew text.
|
20 |
|
21 |
For full details of this model please read our [release blog post](https://example.com).
|
22 |
|
|
|
56 |
## Model Architecture
|
57 |
|
58 |
DictaLM-2.0 is based on the [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) model with the following changes:
|
59 |
+
- An extended tokenizer with 1,000 injected tokens specifically for Hebrew, increasing the compression rate from 5.78 tokens/word to 2.76 tokens/word.
|
60 |
- Continued pretraining on over 190B tokens of naturally occuring text, 50% Hebrew and 50% English.
|
61 |
|
62 |
## Notice
|