sinMT5-tuned

This model is a fine-tuned version of google/mT5 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.8573
• Rouge1: 20.2531
• Rouge2: 8.1307
• Rougel: 19.3917
• Rougelsum: 20.0592

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.00015652249866150822
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 7

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
1.8651	1.0	1500	1.8070	17.676	7.1418	16.8638	17.457
1.5527	2.0	3000	1.7804	21.1357	8.1386	20.122	20.8652
1.3755	3.0	4500	1.7769	21.4151	8.5692	20.3204	21.1152
1.2473	4.0	6000	1.7937	21.2434	8.2325	20.1332	21.0657
1.1548	5.0	7500	1.8035	20.4298	8.2314	19.5909	20.2116
1.0835	6.0	9000	1.8367	20.5427	8.2226	19.6134	20.2918
1.0387	7.0	10500	1.8573	20.2531	8.1307	19.3917	20.0592

Framework versions

• Transformers 4.28.1
• Pytorch 2.0.0+cu118
• Tokenizers 0.13.3