flant5-base

This model is a fine-tuned version of google/flan-t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2172
• Rouge1: 40.5681
• Rouge2: 19.3464
• Rougel: 35.9679
• Rougelsum: 37.6605
• Gen Len: 19.97

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 3
• eval_batch_size: 3
• seed: 42
• gradient_accumulation_steps: 3
• total_train_batch_size: 9
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.293	1.0	1384	0.2280	38.7963	16.8581	33.5577	35.6059	19.96
0.2515	2.0	2769	0.2203	38.8289	16.9039	33.5277	35.3638	19.982
0.2347	3.0	4154	0.2155	39.8194	18.4008	34.6872	36.6037	19.968
0.223	4.0	5538	0.2148	40.1964	18.4087	34.9626	36.8748	19.96
0.2135	5.0	6923	0.2149	40.5411	19.0474	35.5895	37.4265	19.962
0.2056	6.0	8308	0.2146	40.3302	19.1785	35.6352	37.4875	19.962
0.1999	7.0	9692	0.2147	40.0187	19.143	35.3324	37.1204	19.958
0.1948	8.0	11077	0.2159	40.1846	19.1643	35.7202	37.4115	19.966
0.1915	9.0	12462	0.2169	40.7785	19.5846	36.1499	37.9102	19.96
0.1888	10.0	13840	0.2172	40.5681	19.3464	35.9679	37.6605	19.97

Framework versions

• Transformers 4.36.1
• Pytorch 2.1.2
• Datasets 2.19.2
• Tokenizers 0.15.2