medmekk HF staff commited on
Commit
9fe3e7e
1 Parent(s): 481056b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -0
README.md CHANGED
@@ -27,6 +27,12 @@ For a deeper dive into the methods and results, check out our [blog post](https:
27
 
28
  You can easily load and test our model in Transformers. Just follow the code below:
29
 
 
 
 
 
 
 
30
  ```python
31
  model = AutoModelForCausalLM.from_pretrained("HF1BitLLM/Llama3-8B-1.58-Linear-10B-tokens", device_map="cuda", torch_dtype=torch.bfloat16)
32
  tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")
 
27
 
28
  You can easily load and test our model in Transformers. Just follow the code below:
29
 
30
+ Start by installing the transformers version with the correct configuration to load bitnet models
31
+ ```bash
32
+ pip install git+https://github.com/huggingface/transformers.git@refs/pull/33410/head
33
+ ```
34
+
35
+ And then load the model :
36
  ```python
37
  model = AutoModelForCausalLM.from_pretrained("HF1BitLLM/Llama3-8B-1.58-Linear-10B-tokens", device_map="cuda", torch_dtype=torch.bfloat16)
38
  tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")