Update README.md
Browse files
README.md
CHANGED
@@ -36,7 +36,6 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
36 |
import torch
|
37 |
|
38 |
tokenizer = AutoTokenizer.from_pretrained('dicta-il/dictalm-7b-instruct')
|
39 |
-
# If you don't have cuda installed, remove the `.cuda()` call at the end
|
40 |
model = AutoModelForCausalLM.from_pretrained('dicta-il/dictalm-7b-instruct', trust_remote_code=True).cuda()
|
41 |
|
42 |
model.eval()
|
@@ -56,6 +55,11 @@ with torch.inference_mode():
|
|
56 |
print(tokenizer.batch_decode(model.generate(**kwargs), skip_special_tokens=True))
|
57 |
```
|
58 |
|
|
|
|
|
|
|
|
|
|
|
59 |
### Alternative ways to initialize the model:
|
60 |
|
61 |
If you have multiple smaller GPUs, and the package `accelerate` is installed, you can initialize the model split across the devices:
|
@@ -74,12 +78,6 @@ model = AutoModelForCausalLM.from_pretrained('dicta-il/dictalm-7b-instruct', tru
|
|
74 |
```
|
75 |
|
76 |
|
77 |
-
|
78 |
-
There are many different parameters you can input into `kwargs` for different results (greedy, beamsearch, different samplign configurations, longer/shorter respones, etc.).
|
79 |
-
|
80 |
-
You can view the full list of parameters you can pass to the `generate` function [here](https://huggingface.co/docs/transformers/v4.33.0/en/main_classes/text_generation#transformers.GenerationMixin.generate).
|
81 |
-
|
82 |
-
|
83 |
## Citation
|
84 |
|
85 |
If you use DictaLM in your research, please cite ```ADD CITATION HERE```
|
|
|
36 |
import torch
|
37 |
|
38 |
tokenizer = AutoTokenizer.from_pretrained('dicta-il/dictalm-7b-instruct')
|
|
|
39 |
model = AutoModelForCausalLM.from_pretrained('dicta-il/dictalm-7b-instruct', trust_remote_code=True).cuda()
|
40 |
|
41 |
model.eval()
|
|
|
55 |
print(tokenizer.batch_decode(model.generate(**kwargs), skip_special_tokens=True))
|
56 |
```
|
57 |
|
58 |
+
|
59 |
+
There are many different parameters you can input into `kwargs` for different results (greedy, beamsearch, different samplign configurations, longer/shorter respones, etc.).
|
60 |
+
|
61 |
+
You can view the full list of parameters you can pass to the `generate` function [here](https://huggingface.co/docs/transformers/v4.33.0/en/main_classes/text_generation#transformers.GenerationMixin.generate).
|
62 |
+
|
63 |
### Alternative ways to initialize the model:
|
64 |
|
65 |
If you have multiple smaller GPUs, and the package `accelerate` is installed, you can initialize the model split across the devices:
|
|
|
78 |
```
|
79 |
|
80 |
|
|
|
|
|
|
|
|
|
|
|
|
|
81 |
## Citation
|
82 |
|
83 |
If you use DictaLM in your research, please cite ```ADD CITATION HERE```
|