TheBloke commited on
Commit
74c2d55
1 Parent(s): 553912d

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -98,7 +98,7 @@ Models are released as sharded safetensors files.
98
 
99
  | Branch | Bits | GS | AWQ Dataset | Seq Len | Size |
100
  | ------ | ---- | -- | ----------- | ------- | ---- |
101
- | main | 4 | 128 | [wikitext](https://huggingface.co/datasets/wikitext/viewer/wikitext-2-v1/test) | 4096 | 3.89 GB
102
 
103
  <!-- README_AWQ.md-provided-files end -->
104
 
@@ -143,13 +143,13 @@ for output in outputs:
143
 
144
  ### Install the necessary packages
145
 
146
- Requires: AutoAWQ 0.0.2 or later
147
 
148
  ```shell
149
  pip3 install autoawq
150
  ```
151
 
152
- If you have problems installing AutoAWQ using the pre-built wheels, install it from source instead:
153
 
154
  ```shell
155
  pip3 uninstall -y autoawq
@@ -221,7 +221,7 @@ print(pipe(prompt_template)[0]['generated_text'])
221
  <!-- README_AWQ.md-compatibility start -->
222
  ## Compatibility
223
 
224
- The files provided are tested to work with AutoAWQ, and vLLM.
225
 
226
  [Huggingface Text Generation Inference (TGI)](https://github.com/huggingface/text-generation-inference) is not yet compatible with AWQ, but a PR is open which should bring support soon: [TGI PR #781](https://github.com/huggingface/text-generation-inference/issues/781).
227
  <!-- README_AWQ.md-compatibility end -->
 
98
 
99
  | Branch | Bits | GS | AWQ Dataset | Seq Len | Size |
100
  | ------ | ---- | -- | ----------- | ------- | ---- |
101
+ | [main](https://huggingface.co/TheBloke/Llama-2-7b-Chat-AWQ/tree/main) | 4 | 128 | [wikitext](https://huggingface.co/datasets/wikitext/viewer/wikitext-2-v1/test) | 4096 | 3.89 GB
102
 
103
  <!-- README_AWQ.md-provided-files end -->
104
 
 
143
 
144
  ### Install the necessary packages
145
 
146
+ Requires: [AutoAWQ](https://github.com/casper-hansen/AutoAWQ) 0.0.2 or later
147
 
148
  ```shell
149
  pip3 install autoawq
150
  ```
151
 
152
+ If you have problems installing [AutoAWQ](https://github.com/casper-hansen/AutoAWQ) using the pre-built wheels, install it from source instead:
153
 
154
  ```shell
155
  pip3 uninstall -y autoawq
 
221
  <!-- README_AWQ.md-compatibility start -->
222
  ## Compatibility
223
 
224
+ The files provided are tested to work with [AutoAWQ](https://github.com/casper-hansen/AutoAWQ), and [vLLM](https://github.com/vllm-project/vllm).
225
 
226
  [Huggingface Text Generation Inference (TGI)](https://github.com/huggingface/text-generation-inference) is not yet compatible with AWQ, but a PR is open which should bring support soon: [TGI PR #781](https://github.com/huggingface/text-generation-inference/issues/781).
227
  <!-- README_AWQ.md-compatibility end -->