Update README.md
Browse files
README.md
CHANGED
@@ -27,6 +27,12 @@ For a deeper dive into the methods and results, check out our [blog post](https:
|
|
27 |
|
28 |
You can easily load and test our model in Transformers. Just follow the code below:
|
29 |
|
|
|
|
|
|
|
|
|
|
|
|
|
30 |
```python
|
31 |
model = AutoModelForCausalLM.from_pretrained("HF1BitLLM/Llama3-8B-1.58-Linear-10B-tokens", device_map="cuda", torch_dtype=torch.bfloat16)
|
32 |
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")
|
|
|
27 |
|
28 |
You can easily load and test our model in Transformers. Just follow the code below:
|
29 |
|
30 |
+
Start by installing the transformers version with the correct configuration to load bitnet models
|
31 |
+
```bash
|
32 |
+
pip install git+https://github.com/huggingface/transformers.git@refs/pull/33410/head
|
33 |
+
```
|
34 |
+
|
35 |
+
And then load the model :
|
36 |
```python
|
37 |
model = AutoModelForCausalLM.from_pretrained("HF1BitLLM/Llama3-8B-1.58-Linear-10B-tokens", device_map="cuda", torch_dtype=torch.bfloat16)
|
38 |
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")
|