johnrachwanpruna commited on
Commit
3f4c5e7
1 Parent(s): 2a0f3e2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -32,7 +32,7 @@ metrics:
32
  - Join Pruna AI community on Discord [here](https://discord.gg/CP4VSgck) to share feedback/suggestions or get help.
33
 
34
  **Frequently Asked Questions**
35
- - ***How does the compression work?*** The model is compressed by using bitsandbytes.
36
  - ***How does the model quality change?*** The quality of the model output will slightly degrade.
37
  - ***What is the model format?*** We the standard safetensors format.
38
  - ***How to compress my own models?*** You can request premium access to more compression methods and tech support for your specific use-cases [here](https://z0halsaff74.typeform.com/pruna-access?typeform-source=www.pruna.ai).
@@ -44,7 +44,7 @@ from transformers import AutoTokenizer
44
  import transformers
45
  import torch
46
 
47
- model = "PrunaAI/mattshumer-Hermes-2-Pro-11B-GPTQ-4bit"
48
  tokenizer = "mattshumer/Hermes-2-Pro-11B"
49
  messages = [{"role": "user", "content": "What is a large language model?"}]
50
 
 
32
  - Join Pruna AI community on Discord [here](https://discord.gg/CP4VSgck) to share feedback/suggestions or get help.
33
 
34
  **Frequently Asked Questions**
35
+ - ***How does the compression work?*** The model is compressed by using GPTQ.
36
  - ***How does the model quality change?*** The quality of the model output will slightly degrade.
37
  - ***What is the model format?*** We the standard safetensors format.
38
  - ***How to compress my own models?*** You can request premium access to more compression methods and tech support for your specific use-cases [here](https://z0halsaff74.typeform.com/pruna-access?typeform-source=www.pruna.ai).
 
44
  import transformers
45
  import torch
46
 
47
+ model = "PrunaAI/mattshumer-Hermes-2-Pro-11B-GPTQ-8bit"
48
  tokenizer = "mattshumer/Hermes-2-Pro-11B"
49
  messages = [{"role": "user", "content": "What is a large language model?"}]
50