md_mt5_1511_v8

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3865
• Bleu: 0.6474
• Gen Len: 18.7566

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
5.2266	1.0	1250	2.0508	0.8851	16.8074
1.9533	2.0	2500	1.1931	1.1725	18.8668
1.5627	3.0	3750	0.8881	0.6185	18.672
1.237	4.0	5000	0.7267	0.6301	18.7212
1.1021	5.0	6250	0.6370	0.6165	18.679
0.962	6.0	7500	0.5726	0.5885	18.7398
0.8972	7.0	8750	0.5235	0.6011	18.7622
0.8385	8.0	10000	0.4881	0.6087	18.761
0.7949	9.0	11250	0.4579	0.6185	18.7696
0.7482	10.0	12500	0.4342	0.6175	18.755
0.7304	11.0	13750	0.4159	0.631	18.7478
0.7009	12.0	15000	0.4029	0.6373	18.7532
0.6863	13.0	16250	0.3938	0.6434	18.7546
0.6841	14.0	17500	0.3882	0.6464	18.7512
0.6749	15.0	18750	0.3865	0.6474	18.7566

Framework versions

• Transformers 4.35.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1