t5-small-article

This model is a fine-tuned version of google-t5/t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.1645
• Rouge1: 33.3436
• Rouge2: 16.5897
• Rougel: 28.335
• Rougelsum: 31.1228

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-08
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
2.5953	1.0	1880	2.3487	31.0337	15.0267	26.0027	28.8777
2.5437	2.0	3760	2.2894	31.838	15.394	26.7426	29.625
2.5054	3.0	5640	2.2434	32.034	15.5701	26.913	29.8383
2.474	4.0	7520	2.2104	32.0455	15.5593	27.0056	29.8549
2.4456	5.0	9400	2.1876	32.7165	16.1007	27.6665	30.5147
2.4295	6.0	11280	2.1734	33.0987	16.3509	28.011	30.8393
2.4205	7.0	13160	2.1662	33.2262	16.4986	28.1781	30.9915
2.4165	8.0	15040	2.1645	33.3436	16.5897	28.335	31.1228

Framework versions

• Transformers 4.28.0
• Pytorch 1.13.1+cu117
• Datasets 2.18.0
• Tokenizers 0.13.3