---
license: llama2
---


This is Phind v2 QLoRa finetune using my PythonTutor LIMA dataset:
https://huggingface.co/datasets/KrisPi/PythonTutor-LIMA-Finetune

My shy attempt to democratize task-specific, cheap fine-tuning focused around LIMA-like datasets -everybody can afford to generate them (less than 20$) and everybody can finetune them (7 hours in total using 2x3090 GPU ~3$+5$ on vast.ai) 

At the moment of publishing this adapter, there are already production-ready solutions for serving several LorA adapters. I honestly believe that the route of a reproducible, vast collection of adapters on the top of current SOTA models, will enable the open-source community to access GPT-4 level LLMs in the next 12 months.

My main inspirations for this were blazing fast implementation of multi-LORA in Exllamav2 backend, Jon's LMoE and Airoboros dataset, r/LocalLLaMA opinions around models based on LIMA finetunes, and of course the LIMA paper itself.   
To prove the point I'm planning to create a few more finetunes like this, starting with the Airoboros "contextual" category for RAG solutions, adapters for React and DevOps YAML scripting. 

5 epochs, LR=1e-05, batch=2, gradient accumulation 32 (i.e. trying to simulate batch 64), max_len=1024. Rank and Alpha both 128 targeting all modules. trained in bfloat16. Constant schedule, no warm-up.
Flash-Attention 2 turned off due to an issue with batching

Evals:
HumanEval score (2.4 p.p improvement to best Phind v2 score!) for the new prompt:
**{'pass@1': 0.7621951219512195}**
**Base + Extra**
**{'pass@1': 0.7073170731707317}**
Base prompt (0.51 p.p improvement)
{'pass@1': 0.725609756097561}
Base + Extra
{'pass@1': 0.6585365853658537}
Phind v2 with Python Tutor custom prompt is only getting:
{'pass@1': 0.7073170731707317}
Base + Extra
{'pass@1': 0.6463414634146342}
After several HumanEval tests and prompts Phind v2 was maximum able to score: 73.78%
**All evals using Transformers 8bit**

In the long term, I'm planning on experimenting with LIMA + DPO Fine-Tuning, but so far I noticed that LIMA datasets need to be both general and task-specific. The best result I got with around 30% of samples that were task specific. 
https://huggingface.co/datasets/KrisPi/PythonTutor-Evol-1k-DPO-GPT4_vs_35

    r=128,
    lora_alpha=128,
    target_modules=['q_proj','k_proj','v_proj','o_proj','gate_proj','down_proj','up_proj'],
    lora_dropout=0.03,
    
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)