pargraphs_titlesV1.0

This model is a fine-tuned version of t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2697
• Rouge1: 68.705
• Rouge2: 54.5204
• Rougel: 67.7709
• Rougelsum: 67.7942
• Gen Len: 1401169535.5

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 4
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.347	0.44	100	0.2634	65.1158	48.282	63.708	63.7424	1401169536.0
0.2412	0.88	200	0.3167	66.0958	50.4705	65.1041	65.1412	1401169536.0
0.2069	1.32	300	0.2357	68.6707	53.5945	67.3654	67.371	1401169536.0
0.1825	1.76	400	0.3932	65.7022	51.08	64.9927	65.0322	1401169536.0
0.1643	2.2	500	0.2223	69.132	54.5176	67.881	67.8987	1401169535.0
0.1715	2.64	600	0.2227	69.2258	54.2845	68.0181	68.0404	1401169535.5
0.1571	3.08	700	0.2707	68.9908	54.7777	68.1279	68.151	1401169536.0
0.1584	3.52	800	0.2193	70.9126	56.4866	69.6718	69.6687	1401169535.5
0.1565	3.96	900	0.3482	68.6691	54.8446	67.796	67.8541	1401169536.0
0.155	4.4	1000	0.2694	69.1457	55.1123	68.2207	68.2543	1401169536.0
0.1586	4.84	1100	0.2697	68.705	54.5204	67.7709	67.7942	1401169535.5

Framework versions

• Transformers 4.36.0
• Pytorch 2.0.0
• Datasets 2.1.0
• Tokenizers 0.15.0