Edit model card

fine-tuned-flan-t5-20-epochs-wang-lab

This model is a fine-tuned version of google/flan-t5-small on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7769
  • Rouge1: 0.2685
  • Rouge2: 0.0913
  • Rougel: 0.2316
  • Rougelsum: 0.2318
  • Gen Len: 19.51

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 20

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
No log 1.0 301 0.8814 0.2192 0.0734 0.1907 0.1911 19.4
1.4237 2.0 602 0.8212 0.2283 0.0703 0.1962 0.1977 19.39
1.4237 3.0 903 0.7864 0.2324 0.0656 0.1989 0.201 19.35
0.8542 4.0 1204 0.7753 0.2489 0.0789 0.2142 0.2156 19.27
0.7603 5.0 1505 0.7662 0.2458 0.0749 0.2119 0.2131 19.29
0.7603 6.0 1806 0.7608 0.2512 0.0896 0.2203 0.2217 19.48
0.699 7.0 2107 0.7611 0.259 0.0836 0.23 0.2318 19.52
0.699 8.0 2408 0.7566 0.264 0.087 0.2304 0.2312 19.43
0.646 9.0 2709 0.7558 0.2815 0.0947 0.2447 0.2453 19.43
0.6106 10.0 3010 0.7585 0.2692 0.0958 0.2377 0.2374 19.5
0.6106 11.0 3311 0.7611 0.2635 0.0997 0.2283 0.2278 19.5
0.5944 12.0 3612 0.7639 0.2785 0.0928 0.24 0.2403 19.46
0.5944 13.0 3913 0.7612 0.2648 0.0886 0.231 0.2318 19.38
0.5671 14.0 4214 0.7636 0.2786 0.0934 0.2431 0.2428 19.48
0.5387 15.0 4515 0.7656 0.2834 0.0938 0.2442 0.2432 19.47
0.5387 16.0 4816 0.7717 0.278 0.0927 0.2408 0.2403 19.48
0.5349 17.0 5117 0.7725 0.2733 0.0912 0.2367 0.2368 19.47
0.5349 18.0 5418 0.7761 0.2699 0.087 0.2339 0.2332 19.5
0.5182 19.0 5719 0.7786 0.2694 0.0874 0.2333 0.2336 19.5
0.5141 20.0 6020 0.7769 0.2685 0.0913 0.2316 0.2318 19.51

Framework versions

  • Transformers 4.35.2
  • Pytorch 2.1.0+cu121
  • Datasets 2.16.1
  • Tokenizers 0.15.0
Downloads last month
0
Safetensors
Model size
77M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for tanatapanun/fine-tuned-flan-t5-20-epochs-wang-lab

Finetuned
(297)
this model