|
--- |
|
license: llama2 |
|
--- |
|
|
|
|
|
This is Phind v2 QLoRa finetune using my PythonTutor LIMA dataset: |
|
https://huggingface.co/datasets/KrisPi/PythonTutor-LIMA-Finetune |
|
|
|
My shy attempt to democratize task-specific, cheap fine-tuning focused around LIMA-like datasets -everybody can afford to generate them (less than 20$) and everybody can finetune them (7 hours in total using 2x3090 GPU ~3$+5$ on vast.ai) |
|
|
|
At the moment of publishing this adapter, there are already production-ready solutions for serving several LorA adapters. I honestly believe that the route of a reproducible, vast collection of adapters on the top of current SOTA models, will enable the open-source community to access GPT-4 level LLMs in the next 12 months. |
|
|
|
My main inspirations for this were blazing fast implementation of multi-LORA in Exllamav2 backend, Jon's LMoE and Airoboros dataset, r/LocalLLaMA opinions around models based on LIMA finetunes, and of course the LIMA paper itself. |
|
To prove the point I'm planning to create a few more finetunes like this, starting with the Airoboros "contextual" category for RAG solutions, adapters for React and DevOps YAML scripting. |
|
|
|
5 epochs, LR=1e-05, batch=2, gradient accumulation 32 (i.e. trying to simulate batch 64), max_len=1024. Rank and Alpha both 128 targeting all modules. trained in bfloat16. Constant schedule, no warm-up. |
|
Flash-Attention 2 turned off due to an issue with batching |
|
|
|
Evals: |
|
HumanEval score (2.4 p.p improvement to best Phind v2 score!) for the new prompt: |
|
**{'pass@1': 0.7621951219512195}** |
|
**Base + Extra** |
|
**{'pass@1': 0.7073170731707317}** |
|
Base prompt (0.51 p.p improvement) |
|
{'pass@1': 0.725609756097561} |
|
Base + Extra |
|
{'pass@1': 0.6585365853658537} |
|
Phind v2 with Python Tutor custom prompt is only getting: |
|
{'pass@1': 0.7073170731707317} |
|
Base + Extra |
|
{'pass@1': 0.6463414634146342} |
|
After several HumanEval tests and prompts Phind v2 was maximum able to score: 73.78% |
|
**All evals using Transformers 8bit** |
|
|
|
In the long term, I'm planning on experimenting with LIMA + DPO Fine-Tuning, but so far I noticed that LIMA datasets need to be both general and task-specific. The best result I got with around 30% of samples that were task specific. |
|
https://huggingface.co/datasets/KrisPi/PythonTutor-Evol-1k-DPO-GPT4_vs_35 |
|
|
|
r=128, |
|
lora_alpha=128, |
|
target_modules=['q_proj','k_proj','v_proj','o_proj','gate_proj','down_proj','up_proj'], |
|
lora_dropout=0.03, |
|
|
|
bnb_config = BitsAndBytesConfig( |
|
load_in_4bit=True, |
|
bnb_4bit_quant_type="nf4", |
|
bnb_4bit_compute_dtype=torch.bfloat16, |
|
bnb_4bit_use_double_quant=True, |
|
) |
