MiquMaid-v1-70B 6bpw
Description
Exllama quant of NeverSleep/MiquMaid-v1-70B
Other quants:
EXL2: 6bpw, 5bpw, 4bpw, 3.5bpw, 3bpw, 2.4bpw
2.4bpw is probably the most you can fit in a 24gb card
GGUF: 2bit Imatrix GGUF
Custom format:
### Instruction:
{system prompt}
### Input:
{input}
### Response:
{reply}
Contact
Kooten on discord