QTIP Quantized Models
Collection
See https://github.com/Cornell-RelaxML/qtip
•
27 items
•
Updated
•
5
YAML Metadata
Warning:
empty or missing yaml metadata in repo card
(https://huggingface.co/docs/hub/model-cards#model-card-metadata)
This model is compatible with tensor parallelism. The RHT runs per-GPU instead of across GPUs. q, k, v, up, and gate are split along the output channel, and o and down are split along the input channel. This model has slightly worse quality than the non "TP8" model.
- Downloads last month
- 2