zechunliu commited on
Commit
efb73db
1 Parent(s): bbbe9ea

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -40,7 +40,7 @@ We are providing 2 ways to run the model:
40
  To load the pretrained model for further finetuning or evaluation:
41
  ```bash
42
  from transformers import AutoModelForCausalLM, AutoTokenizer
43
- tokenizer = AutoTokenizer.from_pretrained("facebook/MobileLLM-600M")
44
  model = AutoModelForCausalLM.from_pretrained("facebook/MobileLLM-600M", trust_remote_code=True)
45
  ```
46
  Note that the default tokenizer does not contain special tokens. For example you can use:
@@ -64,7 +64,7 @@ We provide the pretraining code in https://github.com/facebookresearch/MobileLLM
64
  # run pretraining
65
  > bash pretrain.sh
66
  ```
67
- We also provide evaluation script for calculating wiki2 testset ppl
68
  ```bash
69
  > bash eval.sh
70
  ```
 
40
  To load the pretrained model for further finetuning or evaluation:
41
  ```bash
42
  from transformers import AutoModelForCausalLM, AutoTokenizer
43
+ tokenizer = AutoTokenizer.from_pretrained("facebook/MobileLLM-600M", use_fast_tokenizer=False)
44
  model = AutoModelForCausalLM.from_pretrained("facebook/MobileLLM-600M", trust_remote_code=True)
45
  ```
46
  Note that the default tokenizer does not contain special tokens. For example you can use:
 
64
  # run pretraining
65
  > bash pretrain.sh
66
  ```
67
+ We also provide evaluation script for calculating ppl of wikitext-2 test split:
68
  ```bash
69
  > bash eval.sh
70
  ```