Update README.md
Browse files
README.md
CHANGED
|
@@ -6,6 +6,8 @@ license: apache-2.0
|
|
| 6 |
|
| 7 |
This model is the Mistral-7b model finetuned for 1k steps with a combined lm loss and distillation loss on Openwebtext2 with a >=20 reddit score filter with training logits from Mixtral. I'm not going to pretend it was a big project I did it in a dream and woke up and replicated the code without any actual reason, idk how well it fares in benchmarks.
|
| 8 |
|
|
|
|
|
|
|
| 9 |
### use
|
| 10 |
```python
|
| 11 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
| 6 |
|
| 7 |
This model is the Mistral-7b model finetuned for 1k steps with a combined lm loss and distillation loss on Openwebtext2 with a >=20 reddit score filter with training logits from Mixtral. I'm not going to pretend it was a big project I did it in a dream and woke up and replicated the code without any actual reason, idk how well it fares in benchmarks.
|
| 8 |
|
| 9 |
+
(update: not very good)
|
| 10 |
+
|
| 11 |
### use
|
| 12 |
```python
|
| 13 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|