Edit model card

qwen2_Magiccoder_evol_10k

This model is a fine-tuned version of Qwen/Qwen2-7B on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7725

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • gradient_accumulation_steps: 16
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 0.02
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss
0.8717 0.0261 4 0.8953
0.8201 0.0522 8 0.8217
0.7952 0.0783 12 0.8046
0.7488 0.1044 16 0.7901
0.8162 0.1306 20 0.7853
0.6904 0.1567 24 0.7835
0.6959 0.1828 28 0.7821
0.8719 0.2089 32 0.7817
0.7546 0.2350 36 0.7795
0.7769 0.2611 40 0.7773
0.838 0.2872 44 0.7758
0.8043 0.3133 48 0.7745
0.7233 0.3395 52 0.7741
0.699 0.3656 56 0.7737
0.728 0.3917 60 0.7731
0.7698 0.4178 64 0.7734
0.8031 0.4439 68 0.7734
0.7362 0.4700 72 0.7731
0.7598 0.4961 76 0.7728
0.7305 0.5222 80 0.7727
0.8435 0.5483 84 0.7726
0.7321 0.5745 88 0.7726
0.8194 0.6006 92 0.7726
0.7417 0.6267 96 0.7728
0.8314 0.6528 100 0.7726
0.711 0.6789 104 0.7726
0.7745 0.7050 108 0.7725
0.744 0.7311 112 0.7724
0.7047 0.7572 116 0.7725
0.6911 0.7834 120 0.7724
0.7593 0.8095 124 0.7725
0.6959 0.8356 128 0.7725
0.8262 0.8617 132 0.7723
0.7802 0.8878 136 0.7724
0.7966 0.9139 140 0.7724
0.7511 0.9400 144 0.7723
0.8079 0.9661 148 0.7724
0.7663 0.9922 152 0.7725

Framework versions

  • PEFT 0.7.1
  • Transformers 4.40.2
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.1
  • Tokenizers 0.19.1
Downloads last month
2
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for imdatta0/qwen2_Magiccoder_evol_10k

Base model

Qwen/Qwen2-7B
Adapter
(233)
this model