ExLlamaV2 quantizations
Collection
All my EXL2 quants here.
•
32 items
•
Updated
This is an ExLlamaV2 quantized model in 4bpw of mpasila/JP-EN-Translator-1K-steps-7B-merged using the default calibration dataset.
Experimental model, may not perform that well. Dataset used is a modified version of NilanE/ParallelFiction-Ja_En-100k.
Next version should be better (I'll use a GPU with more memory since the dataset happens to use pretty long samples).
Below is a translation task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
{}
### Input:
{}
### Response:
{}
This mistral model was trained 2x faster with Unsloth and Huggingface's TRL library.
Base model
augmxnt/shisa-base-7b-v1