md_mt5_0109_v6

This model is a fine-tuned version of Buseak/md_mt5_0109_v5 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0628
• Bleu: 0.6537
• Gen Len: 18.9513

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
0.1911	1.0	975	0.0801	0.6356	18.9449
0.1854	2.0	1950	0.0782	0.6365	18.9446
0.1807	3.0	2925	0.0755	0.6419	18.9485
0.175	4.0	3900	0.0732	0.6431	18.949
0.1699	5.0	4875	0.0720	0.6471	18.949
0.1669	6.0	5850	0.0701	0.6474	18.9497
0.165	7.0	6825	0.0682	0.6494	18.95
0.1604	8.0	7800	0.0673	0.6508	18.9505
0.1585	9.0	8775	0.0665	0.6516	18.9505
0.1512	10.0	9750	0.0652	0.6518	18.9508
0.1543	11.0	10725	0.0646	0.653	18.9505
0.155	12.0	11700	0.0639	0.6533	18.9505
0.1506	13.0	12675	0.0633	0.6537	18.951
0.1493	14.0	13650	0.0629	0.6538	18.951
0.1486	15.0	14625	0.0628	0.6537	18.9513

Framework versions

• Transformers 4.38.1
• Pytorch 2.1.0+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2