Update README.md
Browse files
README.md
CHANGED
|
@@ -27,7 +27,13 @@ For a deeper dive into the methods and results, check out our [blog post](https:
|
|
| 27 |
|
| 28 |
You can easily load and test our model in Transformers. Just follow the code below:
|
| 29 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
```python
|
|
|
|
| 31 |
model = AutoModelForCausalLM.from_pretrained("HF1BitLLM/Llama3-8B-1.58-100B-tokens", device_map="cuda", torch_dtype=torch.bfloat16)
|
| 32 |
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")
|
| 33 |
|
|
|
|
| 27 |
|
| 28 |
You can easily load and test our model in Transformers. Just follow the code below:
|
| 29 |
|
| 30 |
+
Start by installing the transformers version with the correct configuration to load bitnet models
|
| 31 |
+
```bash
|
| 32 |
+
pip install git+https://github.com/huggingface/transformers.git@refs/pull/33410/head
|
| 33 |
+
```
|
| 34 |
+
And then load the model :
|
| 35 |
```python
|
| 36 |
+
|
| 37 |
model = AutoModelForCausalLM.from_pretrained("HF1BitLLM/Llama3-8B-1.58-100B-tokens", device_map="cuda", torch_dtype=torch.bfloat16)
|
| 38 |
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")
|
| 39 |
|