Llama-3.1 Quantization Collection Neural Magic quantized Llama-3.1 models • 21 items • Updated 8 days ago • 35
FP8 LLMs for vLLM Collection Accurate FP8 quantized models by Neural Magic, ready for use with vLLM! • 43 items • Updated 7 days ago • 53
QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models Paper • 2310.16795 • Published Oct 25, 2023 • 26