t5-abs-2209-2245-lr-0.0001-bs-10-maxep-10

This model is a fine-tuned version of google-t5/t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.6999
• Rouge/rouge1: 0.4672
• Rouge/rouge2: 0.2204
• Rouge/rougel: 0.3985
• Rouge/rougelsum: 0.4
• Bertscore/bertscore-precision: 0.8972
• Bertscore/bertscore-recall: 0.8951
• Bertscore/bertscore-f1: 0.896
• Meteor: 0.4246
• Gen Len: 38.8545

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 10
• eval_batch_size: 10
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 20
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge/rouge1	Rouge/rouge2	Rouge/rougel	Rouge/rougelsum	Bertscore/bertscore-precision	Bertscore/bertscore-recall	Bertscore/bertscore-f1	Meteor	Gen Len
2.6121	0.9885	43	1.9135	0.4781	0.2192	0.4127	0.4144	0.9061	0.8909	0.8983	0.4041	32.2364
1.8386	2.0	87	1.7719	0.4589	0.2133	0.3883	0.3891	0.8976	0.8919	0.8946	0.4147	37.3182
1.7081	2.9885	130	1.7388	0.4565	0.2189	0.3905	0.3913	0.8962	0.8923	0.8941	0.4138	37.9545
1.5451	4.0	174	1.7229	0.4554	0.2107	0.3835	0.3858	0.8955	0.8927	0.8939	0.4167	38.9636
1.4864	4.9885	217	1.7190	0.4599	0.2167	0.3928	0.3951	0.897	0.8939	0.8953	0.4216	38.5636
1.397	6.0	261	1.7236	0.4539	0.2174	0.3892	0.3908	0.8964	0.8932	0.8946	0.416	38.3182
1.3893	6.9885	304	1.7349	0.4582	0.213	0.3905	0.3921	0.8952	0.8948	0.8949	0.4222	39.2818
1.3211	8.0	348	1.7075	0.4622	0.2159	0.3934	0.3948	0.8962	0.8938	0.8949	0.4211	38.9545
1.4086	8.9885	391	1.6987	0.4675	0.22	0.3976	0.3995	0.8971	0.8947	0.8958	0.4243	38.7182
1.3235	9.8851	430	1.6999	0.4672	0.2204	0.3985	0.4	0.8972	0.8951	0.896	0.4246	38.8545

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1