Update README.md
Browse files
README.md
CHANGED
@@ -40,7 +40,7 @@ We are providing 2 ways to run the model:
|
|
40 |
To load the pretrained model for further finetuning or evaluation:
|
41 |
```bash
|
42 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
43 |
-
tokenizer = AutoTokenizer.from_pretrained("facebook/MobileLLM-600M")
|
44 |
model = AutoModelForCausalLM.from_pretrained("facebook/MobileLLM-600M", trust_remote_code=True)
|
45 |
```
|
46 |
Note that the default tokenizer does not contain special tokens. For example you can use:
|
@@ -64,7 +64,7 @@ We provide the pretraining code in https://github.com/facebookresearch/MobileLLM
|
|
64 |
# run pretraining
|
65 |
> bash pretrain.sh
|
66 |
```
|
67 |
-
We also provide evaluation script for calculating
|
68 |
```bash
|
69 |
> bash eval.sh
|
70 |
```
|
|
|
40 |
To load the pretrained model for further finetuning or evaluation:
|
41 |
```bash
|
42 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
43 |
+
tokenizer = AutoTokenizer.from_pretrained("facebook/MobileLLM-600M", use_fast_tokenizer=False)
|
44 |
model = AutoModelForCausalLM.from_pretrained("facebook/MobileLLM-600M", trust_remote_code=True)
|
45 |
```
|
46 |
Note that the default tokenizer does not contain special tokens. For example you can use:
|
|
|
64 |
# run pretraining
|
65 |
> bash pretrain.sh
|
66 |
```
|
67 |
+
We also provide evaluation script for calculating ppl of wikitext-2 test split:
|
68 |
```bash
|
69 |
> bash eval.sh
|
70 |
```
|