Qwen 2.5 Math 14B Iter 2
Collection
Qwen 2.5 is missing it's 14B and 32B math variants!! I have taken it upon myself to create them :) These are the Iteration 2 Models
•
4 items
•
Updated
•
1
Uploaded model
- Developed by: qingy2019
- License: apache-2.0
- Finetuned from model : unsloth/qwen2.5-14b-bnb-4bit
Huge thanks to Unsloth and the Huggingface TRL library.
This model is Qwen 2.5 14B fine tuned for a full epoch on the high quality garage-bAInd/Open-Platypus dataset for STEM reasoning.
| Training Detail | Value |
|---|---|
| Epochs | 1 |
| Steps | 2077 |
| Loss | 0.4218 |
| Batch size | 4 |
| Gradient Acc. Steps | 3 |
| Learning Rate | 2e-4 |
| LR Scheduler | cosine |
| Rank | 32 |
| Rank-Stabilized LoRA | Yes |
| Warm up steps | 5 |
| Weight Decay | 0.01 |
| Seed | 3407 |
