Made by merging the following lora:
https://huggingface.co/Neko-Institute-of-Science/VicUnLocked-30b-LoRA

Then quantizing with ooba's old CUDA branch of GPTQ
```
python llama.py vicunlocked-30b c4 --wbits 4 --true-sequential --act-order --save_safetensors vicunlocked-30b-4bit.safetensors
```
{}
transformers pytorch llama text-generation autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #llama #text-generation #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

Made by merging the following lora:
URL

Then quantizing with ooba's old CUDA branch of GPTQ

TAGS
#transformers #pytorch #llama #text-generation #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
