flan-t5-base-cnn_dailymail

This model is a fine-tuned version of t5-small on the cnn_dailymail dataset. It achieves the following results on the evaluation set:

• Loss: 1.8013
• Rouge1: 24.6545
• Rouge2: 11.7282
• Rougel: 20.3578
• Rougelsum: 23.1966
• Gen Len: 18.9989

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.0058	1.0	17945	1.8259	24.6279	11.6692	20.3361	23.1875	18.9988
1.97	2.0	35890	1.8158	24.6935	11.7554	20.4015	23.2584	18.9985
1.962	3.0	53835	1.8095	24.6151	11.7178	20.3361	23.1781	18.9993
1.9551	4.0	71780	1.8040	24.6127	11.7364	20.3473	23.17	18.9989
1.9515	5.0	89725	1.8013	24.6545	11.7282	20.3578	23.1966	18.9989

Framework versions

• Transformers 4.29.1
• Pytorch 2.0.1+cu117
• Datasets 2.12.0
• Tokenizers 0.13.3