learn3r's picture
End of training
68d715b verified
metadata
license: apache-2.0
base_model: google/long-t5-tglobal-xl
tags:
  - generated_from_trainer
datasets:
  - learn3r/summ_screen_memsum_oracle
model-index:
  - name: longt5_xl_sfd_memsum_30
    results: []

longt5_xl_sfd_memsum_30

This model is a fine-tuned version of google/long-t5-tglobal-xl on the learn3r/summ_screen_memsum_oracle dataset. It achieves the following results on the evaluation set:

  • Loss: 5.1322

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.001
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 32
  • total_train_batch_size: 256
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: constant
  • num_epochs: 30.0

Training results

Training Loss Epoch Step Validation Loss
2.6697 0.97 14 2.4168
2.2272 1.95 28 2.2644
1.9024 2.99 43 2.2556
1.6554 3.97 57 2.4007
1.3619 4.94 71 2.4233
1.1577 5.98 86 2.6797
0.9584 6.96 100 2.8449
0.7197 8.0 115 3.0255
0.5756 8.97 129 3.1467
0.485 9.95 143 3.2976
0.4027 10.99 158 3.8111
0.2938 11.97 172 3.7330
0.2665 12.94 186 4.1417
0.2019 13.98 201 4.0316
0.1706 14.96 215 4.1357
0.1418 16.0 230 4.1022
0.1286 16.97 244 4.1198
0.1022 17.95 258 4.1862
0.1122 18.99 273 4.6386
0.093 19.97 287 4.6829
0.0783 20.94 301 4.6637
0.0698 21.98 316 4.7190
0.0688 22.96 330 5.0200
0.0633 24.0 345 4.7576
0.0609 24.97 359 4.7805
0.0553 25.95 373 4.7338
0.0503 26.99 388 5.1409
0.0471 27.97 402 5.1463
0.0472 28.94 416 5.1636
0.0376 29.22 420 5.1322

Framework versions

  • Transformers 4.36.2
  • Pytorch 2.1.2+cu121
  • Datasets 2.16.1
  • Tokenizers 0.15.0