pargraphs_titles

This model is a fine-tuned version of t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2408
• Rouge1: 68.8334
• Rouge2: 59.1001
• Rougel: 67.5283
• Rougelsum: 67.6312
• Gen Len: 13.7977

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 4
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	455	0.4086	56.8689	47.8503	56.3172	56.1544	13.7393
0.2169	2.0	910	0.1848	66.7857	58.5467	65.8133	65.7037	13.8604
0.1629	3.0	1365	0.1768	67.5534	59.4926	66.5736	66.5271	13.7222
0.1347	4.0	1820	0.2881	67.7423	58.9397	66.4095	66.5328	13.8362
0.1277	5.0	2275	0.2408	68.8334	59.1001	67.5283	67.6312	13.7977

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Tokenizers 0.15.0