fine-tuned-flan-t5-20-epochs

This model is a fine-tuned version of google/flan-t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7842
• Rouge1: 0.2614
• Rouge2: 0.0824
• Rougel: 0.226
• Rougelsum: 0.2273
• Gen Len: 14.54

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	301	1.8551	0.1314	0.0425	0.1139	0.1139	11.4
2.7128	2.0	602	0.9826	0.1868	0.065	0.1564	0.1571	15.06
2.7128	3.0	903	0.8569	0.2079	0.0718	0.1716	0.1722	15.05
1.1113	4.0	1204	0.8300	0.2141	0.0705	0.181	0.181	14.59
0.9116	5.0	1505	0.8204	0.2254	0.0837	0.1943	0.1945	14.92
0.9116	6.0	1806	0.8116	0.243	0.0807	0.2074	0.2072	15.03
0.8732	7.0	2107	0.8082	0.2376	0.0752	0.2015	0.2016	14.83
0.8732	8.0	2408	0.8007	0.2345	0.0735	0.2015	0.2021	14.41
0.8336	9.0	2709	0.7968	0.2456	0.0757	0.2081	0.2081	14.4
0.8151	10.0	3010	0.7942	0.2544	0.0752	0.2134	0.2146	14.58
0.8151	11.0	3311	0.7924	0.2497	0.0783	0.2118	0.2124	14.5
0.8187	12.0	3612	0.7907	0.2552	0.0769	0.2189	0.2191	14.43
0.8187	13.0	3913	0.7891	0.258	0.077	0.2197	0.2199	14.37
0.8028	14.0	4214	0.7867	0.2511	0.0801	0.2146	0.2147	14.71
0.7793	15.0	4515	0.7852	0.2551	0.0777	0.2175	0.2177	14.67
0.7793	16.0	4816	0.7858	0.2594	0.0774	0.2219	0.2219	14.47
0.7872	17.0	5117	0.7850	0.2609	0.0803	0.2233	0.2244	14.56
0.7872	18.0	5418	0.7843	0.2599	0.0811	0.2242	0.2256	14.55
0.7756	19.0	5719	0.7844	0.261	0.0824	0.2256	0.2271	14.55
0.7752	20.0	6020	0.7842	0.2614	0.0824	0.226	0.2273	14.54

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.0
• Tokenizers 0.15.0