veronica-girolimetti/t5-summarization-one-shot-20-epochs

test-dialogue-summarization

This model is a fine-tuned version of google/flan-t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2304
• Rouge: {'rouge1': 47.6559, 'rouge2': 23.5195, 'rougeL': 21.653, 'rougeLsum': 21.653}
• Bert Score: 0.8778
• Bleurt 20: -0.769
• Gen Len: 16.205

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 7
• eval_batch_size: 7
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge	Bert Score	Bleurt 20	Gen Len
2.7518	1.0	186	2.4544	{'rouge1': 42.0552, 'rouge2': 18.6296, 'rougeL': 20.1713, 'rougeLsum': 20.1713}	0.8684	-0.8842	16.4
2.5043	2.0	372	2.3359	{'rouge1': 44.4236, 'rouge2': 20.2933, 'rougeL': 20.781, 'rougeLsum': 20.781}	0.8694	-0.858	17.06
2.3625	3.0	558	2.2849	{'rouge1': 42.6795, 'rouge2': 19.7272, 'rougeL': 20.5673, 'rougeLsum': 20.5673}	0.8724	-0.8485	16.0
2.1931	4.0	744	2.2602	{'rouge1': 46.2739, 'rouge2': 21.51, 'rougeL': 21.0248, 'rougeLsum': 21.0248}	0.8749	-0.8192	16.085
2.1187	5.0	930	2.2430	{'rouge1': 44.6328, 'rouge2': 21.1871, 'rougeL': 20.8, 'rougeLsum': 20.8}	0.8729	-0.8465	16.475
2.0406	6.0	1116	2.2199	{'rouge1': 43.9237, 'rouge2': 21.0488, 'rougeL': 20.7538, 'rougeLsum': 20.7538}	0.8724	-0.8379	16.195
2.0104	7.0	1302	2.2111	{'rouge1': 45.6132, 'rouge2': 21.7648, 'rougeL': 21.0134, 'rougeLsum': 21.0134}	0.8738	-0.8203	16.175
1.9662	8.0	1488	2.2029	{'rouge1': 44.6747, 'rouge2': 21.4751, 'rougeL': 20.9398, 'rougeLsum': 20.9398}	0.8728	-0.8446	16.2
1.8518	9.0	1674	2.2129	{'rouge1': 46.7682, 'rouge2': 22.4301, 'rougeL': 22.1849, 'rougeLsum': 22.1849}	0.877	-0.7737	16.445
1.8581	10.0	1860	2.2144	{'rouge1': 46.788, 'rouge2': 22.5919, 'rougeL': 21.9838, 'rougeLsum': 21.9838}	0.8766	-0.7886	16.175
1.805	11.0	2046	2.2126	{'rouge1': 46.41, 'rouge2': 22.3295, 'rougeL': 21.6966, 'rougeLsum': 21.6966}	0.8771	-0.7902	16.08
1.766	12.0	2232	2.2228	{'rouge1': 48.3228, 'rouge2': 23.2358, 'rougeL': 22.2037, 'rougeLsum': 22.2037}	0.8778	-0.7648	16.42
1.7661	13.0	2418	2.2235	{'rouge1': 47.3602, 'rouge2': 23.0001, 'rougeL': 22.0806, 'rougeLsum': 22.0806}	0.8772	-0.7872	16.205
1.689	14.0	2604	2.2284	{'rouge1': 46.8864, 'rouge2': 22.952, 'rougeL': 21.6138, 'rougeLsum': 21.6138}	0.8784	-0.7702	16.015
1.7035	15.0	2790	2.2165	{'rouge1': 47.1586, 'rouge2': 23.3426, 'rougeL': 21.471, 'rougeLsum': 21.471}	0.8789	-0.7622	15.945
1.7013	16.0	2976	2.2215	{'rouge1': 47.0545, 'rouge2': 22.962, 'rougeL': 21.5717, 'rougeLsum': 21.5717}	0.879	-0.7537	15.995
1.6886	17.0	3162	2.2276	{'rouge1': 47.3071, 'rouge2': 23.0284, 'rougeL': 21.5429, 'rougeLsum': 21.5429}	0.8781	-0.758	16.07
1.6828	18.0	3348	2.2273	{'rouge1': 47.2229, 'rouge2': 22.9743, 'rougeL': 21.756, 'rougeLsum': 21.756}	0.8777	-0.7784	16.12
1.6164	19.0	3534	2.2286	{'rouge1': 47.4937, 'rouge2': 23.2693, 'rougeL': 21.7418, 'rougeLsum': 21.7418}	0.8771	-0.7742	16.225
1.6247	20.0	3720	2.2304	{'rouge1': 47.6559, 'rouge2': 23.5195, 'rougeL': 21.653, 'rougeLsum': 21.653}	0.8778	-0.769	16.205

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0