TheBloke commited on
Commit
ffd46b4
1 Parent(s): d5488b3

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -63,7 +63,7 @@ This is a Mixtral AWQ model.
63
 
64
  For AutoAWQ inference, please install AutoAWQ 0.1.8 or later.
65
 
66
- Support via Transformers is coming soon, via this PR: https://github.com/huggingface/transformers/pull/27950 which should be merged to Transformers `main` very soon.
67
 
68
  vLLM: version 0.2.6 is confirmed to support Mixtral AWQs.
69
 
@@ -113,7 +113,7 @@ Models are released as sharded safetensors files.
113
 
114
  | Branch | Bits | GS | AWQ Dataset | Seq Len | Size |
115
  | ------ | ---- | -- | ----------- | ------- | ---- |
116
- | [main](https://huggingface.co/TheBloke/SauerkrautLM-Mixtral-8x7B-Instruct-AWQ/tree/main) | 4 | 128 | [VMware Open Instruct](https://huggingface.co/datasets/VMware/open-instruct/viewer/) | 8192 | 24.65 GB
117
 
118
  <!-- README_AWQ.md-provided-files end -->
119
 
 
63
 
64
  For AutoAWQ inference, please install AutoAWQ 0.1.8 or later.
65
 
66
+ Support via Transformers is also available, but currently requires installing Transformers from Github: `pip3 install git+https://github.com/huggingface/transformers.git`
67
 
68
  vLLM: version 0.2.6 is confirmed to support Mixtral AWQs.
69
 
 
113
 
114
  | Branch | Bits | GS | AWQ Dataset | Seq Len | Size |
115
  | ------ | ---- | -- | ----------- | ------- | ---- |
116
+ | [main](https://huggingface.co/TheBloke/SauerkrautLM-Mixtral-8x7B-Instruct-AWQ/tree/main) | 4 | 128 | [German Quad](https://huggingface.co/datasets/deepset/germanquad/viewer/) | 8192 | 24.65 GB
117
 
118
  <!-- README_AWQ.md-provided-files end -->
119