Anhforth commited on
Commit
765d32a
1 Parent(s): 15e4261

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -31,8 +31,10 @@ The additional details of the Aquila model will be presented in the official tec
31
  ### 1. Inference
32
 
33
  ```python
34
- from transformers import AutoTokenizer, AutoModelForCausalLM
35
  import torch
 
 
 
36
  device = torch.device("cuda")
37
  model_info = "BAAI/Aquila2-34B"
38
  tokenizer = AutoTokenizer.from_pretrained(model_info, trust_remote_code=True)
 
31
  ### 1. Inference
32
 
33
  ```python
 
34
  import torch
35
+ from transformers import AutoTokenizer, AutoModelForCausalLM
36
+ from transformers import BitsAndBytesConfig
37
+
38
  device = torch.device("cuda")
39
  model_info = "BAAI/Aquila2-34B"
40
  tokenizer = AutoTokenizer.from_pretrained(model_info, trust_remote_code=True)