Edit model card

peft-dialogue-summary-training-1716404262

This model is a fine-tuned version of ibm-granite/granite-8b-code-instruct on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.5224

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0002
  • train_batch_size: 1
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 4
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 1
  • training_steps: 400

Training results

Training Loss Epoch Step Validation Loss
7.625 0.1453 25 5.7401
5.3519 0.2907 50 4.5831
4.2545 0.4360 75 3.8334
4.2506 0.5814 100 3.3488
3.5118 0.7267 125 2.8198
3.1713 0.8721 150 2.4324
2.5504 1.0174 175 2.3330
2.174 1.1628 200 2.2412
1.878 1.3081 225 2.1900
1.9039 1.4535 250 2.0439
1.7977 1.5988 275 1.9491
1.7755 1.7442 300 1.8278
1.629 1.8895 325 1.6587
1.5533 2.0349 350 1.5799
1.2937 2.1802 375 1.5344
1.1375 2.3256 400 1.5224

Framework versions

  • PEFT 0.11.1
  • Transformers 4.41.0
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.1
  • Tokenizers 0.19.1
Downloads last month
0
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for sinanazeri/peft-dialogue-summary-training-1716404262