malhajar commited on
Commit
269b040
1 Parent(s): 50776a3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -2
README.md CHANGED
@@ -36,11 +36,16 @@ will report soon
36
  `garage-bAInd/Platypus2-70B` was instruction fine-tuned using gptq on 2 L40 48GB.
37
 
38
  ## How to Get Started with the Model
 
 
 
 
39
 
40
  Use the code sample provided in the original post to interact with the model.
41
  ```python
42
- from transformers import AutoModelForCausalLM, AutoTokenizer
43
-
 
44
  model_id = "malhajar/Platypus2-70B-instruct-4bit-gptq"
45
  model = AutoGPTQForCausalLM.from_quantized(model_id,inject_fused_attention=False,
46
  use_safetensors=True,
 
36
  `garage-bAInd/Platypus2-70B` was instruction fine-tuned using gptq on 2 L40 48GB.
37
 
38
  ## How to Get Started with the Model
39
+ First install auto_gptq with
40
+ ```shell
41
+ pip install auto_gptq
42
+ ```
43
 
44
  Use the code sample provided in the original post to interact with the model.
45
  ```python
46
+ from transformers import AutoTokenizer
47
+ from auto_gptq import AutoGPTQForCausalLM
48
+
49
  model_id = "malhajar/Platypus2-70B-instruct-4bit-gptq"
50
  model = AutoGPTQForCausalLM.from_quantized(model_id,inject_fused_attention=False,
51
  use_safetensors=True,