md_mt5_2311_retrain_v19_iwt

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6818
• Bleu: 1.1458
• Gen Len: 18.8547

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
14.6802	1.0	635	5.8279	0.001	11.8701
6.5233	2.0	1270	2.4491	1.4911	15.5764
3.4983	3.0	1905	1.6292	2.1885	17.7457
2.1412	4.0	2540	1.3098	2.7626	18.6941
1.8999	5.0	3175	1.1263	1.0375	18.4476
1.7456	6.0	3810	0.9967	1.172	18.7098
1.6077	7.0	4445	0.8971	1.2144	18.7425
1.4144	8.0	5080	0.8289	1.1994	18.7866
1.342	9.0	5715	0.7807	1.1842	18.8138
1.3029	10.0	6350	0.7468	1.17	18.8524
1.2426	11.0	6985	0.7213	1.1513	18.8488
1.1793	12.0	7620	0.7024	1.1573	18.8516
1.1649	13.0	8255	0.6909	1.1472	18.8524
1.1419	14.0	8890	0.6837	1.1483	18.8535
1.1289	15.0	9525	0.6818	1.1458	18.8547

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu118
• Datasets 2.15.0
• Tokenizers 0.15.0