mychen76
/

Llama-3.1-8B-bonito-v1-awq

4-bit precision

Model card Files Files and versions Community

mychen76 commited on Sep 17

Commit

4c9cacd

•

1 Parent(s): ebdf537

Update README.md

Files changed (1) hide show

README.md +5 -0

README.md CHANGED Viewed

@@ -33,6 +33,11 @@ SHORTFORM_TO_FULL_TASK_TYPES = {
 Bonito AWQ Usage Notebook [AWQ_Inference-Bonito.ipynb](https://huggingface.co/mychen76/Llama-3.1-8B-bonito-v1-awq/blob/main/AWQ_Inference-Bonito.ipynb)
 ### Using Quantized Bonito
 ```

 Bonito AWQ Usage Notebook [AWQ_Inference-Bonito.ipynb](https://huggingface.co/mychen76/Llama-3.1-8B-bonito-v1-awq/blob/main/AWQ_Inference-Bonito.ipynb)
+include custom class:
+- AWQBonito (inference based on AutoAWQForCausalLM)
+- VLLMBonito (inference based on vLLM)
 ### Using Quantized Bonito
 ```