samzirbo/mt5.gendered.old

mt5.gendered

This model is a fine-tuned version of samzirbo/mT5.en-es.pretrained on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.5115
• Bleu: 38.4985
• Meteor: 0.6605
• Chrf++: 60.6076

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 1000
• training_steps: 30000

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Meteor	Chrf++
4.0658	0.3215	3000	2.1146	29.6159	0.587	53.1876
2.3311	0.6431	6000	1.8553	33.4209	0.6198	56.5789
2.0997	0.9646	9000	1.7308	35.2211	0.6359	58.0995
1.9354	1.2862	12000	1.6645	36.311	0.6437	58.9512
1.8583	1.6077	15000	1.6007	37.0671	0.6512	59.446
1.8043	1.9293	18000	1.5639	37.7823	0.6553	60.0906
1.728	2.2508	21000	1.5318	37.9166	0.6565	60.1504
1.6989	2.5723	24000	1.5188	38.4261	0.6603	60.5626
1.6854	2.8939	27000	1.5115	38.5249	0.6607	60.6204
1.6696	3.2154	30000	1.5115	38.4985	0.6605	60.6076

Framework versions

• Transformers 4.40.1
• Pytorch 2.2.1+cu121
• Datasets 2.19.0
• Tokenizers 0.19.1