sMPNG_t5_base_test

This model is a fine-tuned version of t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3948
• Bleu: 61.2155
• Gen Len: 7.3325

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
0.889	1.0	5246	0.7274	43.8011	7.0999
0.7403	2.0	10492	0.6177	48.5778	7.2271
0.6483	3.0	15738	0.5612	51.6132	7.131
0.5839	4.0	20984	0.5213	53.6657	7.2675
0.5613	5.0	26230	0.4986	55.0196	7.2437
0.5172	6.0	31476	0.4797	56.0442	7.3339
0.4929	7.0	36722	0.4588	56.7114	7.3136
0.4786	8.0	41968	0.4452	57.7053	7.3036
0.4382	9.0	47214	0.4341	57.9617	7.2889
0.4214	10.0	52460	0.4268	58.3971	7.3519
0.4187	11.0	57706	0.4178	58.9582	7.3124
0.397	12.0	62952	0.4115	59.5624	7.2993
0.3848	13.0	68198	0.4049	60.4898	7.3177
0.3635	14.0	73444	0.4037	60.5544	7.3293
0.3669	15.0	78690	0.4015	60.8635	7.3081
0.3675	16.0	83936	0.3968	60.8527	7.3213
0.3548	17.0	89182	0.3974	60.8292	7.3292
0.3534	18.0	94428	0.3944	61.1633	7.3402
0.35	19.0	99674	0.3947	61.1179	7.3329
0.3299	20.0	104920	0.3948	61.2155	7.3325

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.1