chansung

/

alpaca-lora-30b

like 50

It was trained on a initial version of cleaned dataset from tloen/alpaca-lora. I will refresh once the more of clean ups are done

Awesome, from trying multiple versions I can tell that the cleaned data has a huge effect. The original Alpaca (not cleaned) would say "As a large language model blah blah consider the many factors blah blah", but the cleaned one does it a lot less. The final cleaned set should be even better good.

I've been running the 30B model at 4bit (https://github.com/johnsmith0031/alpaca_lora_4bit) and it's fantastic to have it on my desktop.

There are some interesting data sets out there (in order of quality)

• oasst 20k (pre-released from open assistant, see discord)
• https://huggingface.co/datasets/stanfordnlp/SHP 300k
• https://github.com/hendrycks/ethics
• https://huggingface.co/datasets/Anthropic/hh-rlhf
• https://github.com/allenai/natural-instructions 64k
• https://github.com/orhonovich/unnatural-instructions
• https://laion.ai/blog/oig-dataset/
• https://huggingface.co/datasets/HuggingFaceH4/stack-exchange-preferences

Now I'm not suggesting you used them, I'm just sharing them out of interest. But the fine tuning approach from Anthropic seems to be

1. fine tune on a large low quality instruction dataset
2. fine tune on a small high quality instruction dataset

But here we are skipping straight to a high quality dataset, and it works OK.

The model is refreshed once again just today

Thanks, there seems to be a quite a difference! It's quite fun to play with

I want to fine tune a alpaca30b with your finetuned config on my custom dataset. I tried this:
model = LlamaForCausalLM.from_pretrained(
base_model,
load_in_8bit=True,
device_map=device_map,
)
model = PeftModel.from_pretrained(model, "chansung/alpaca-lora-30b")
tokenizer = LlamaTokenizer.from_pretrained(base_model)

My scripts is just finetune.py from alpacalora, with that peft addition.

Training runs, but during inference it doesnt seem to have any of your fine tuning present. Can you offer any help. Thanks for your work.