johnrachwanpruna
commited on
Commit
•
3f4c5e7
1
Parent(s):
2a0f3e2
Update README.md
Browse files
README.md
CHANGED
@@ -32,7 +32,7 @@ metrics:
|
|
32 |
- Join Pruna AI community on Discord [here](https://discord.gg/CP4VSgck) to share feedback/suggestions or get help.
|
33 |
|
34 |
**Frequently Asked Questions**
|
35 |
-
- ***How does the compression work?*** The model is compressed by using
|
36 |
- ***How does the model quality change?*** The quality of the model output will slightly degrade.
|
37 |
- ***What is the model format?*** We the standard safetensors format.
|
38 |
- ***How to compress my own models?*** You can request premium access to more compression methods and tech support for your specific use-cases [here](https://z0halsaff74.typeform.com/pruna-access?typeform-source=www.pruna.ai).
|
@@ -44,7 +44,7 @@ from transformers import AutoTokenizer
|
|
44 |
import transformers
|
45 |
import torch
|
46 |
|
47 |
-
model = "PrunaAI/mattshumer-Hermes-2-Pro-11B-GPTQ-
|
48 |
tokenizer = "mattshumer/Hermes-2-Pro-11B"
|
49 |
messages = [{"role": "user", "content": "What is a large language model?"}]
|
50 |
|
|
|
32 |
- Join Pruna AI community on Discord [here](https://discord.gg/CP4VSgck) to share feedback/suggestions or get help.
|
33 |
|
34 |
**Frequently Asked Questions**
|
35 |
+
- ***How does the compression work?*** The model is compressed by using GPTQ.
|
36 |
- ***How does the model quality change?*** The quality of the model output will slightly degrade.
|
37 |
- ***What is the model format?*** We the standard safetensors format.
|
38 |
- ***How to compress my own models?*** You can request premium access to more compression methods and tech support for your specific use-cases [here](https://z0halsaff74.typeform.com/pruna-access?typeform-source=www.pruna.ai).
|
|
|
44 |
import transformers
|
45 |
import torch
|
46 |
|
47 |
+
model = "PrunaAI/mattshumer-Hermes-2-Pro-11B-GPTQ-8bit"
|
48 |
tokenizer = "mattshumer/Hermes-2-Pro-11B"
|
49 |
messages = [{"role": "user", "content": "What is a large language model?"}]
|
50 |
|