mms-1b-bem-male-sv

This model is a fine-tuned version of facebook/mms-1b-all on the BEMBASPEECH - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.1409
• Wer: 0.3498

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 5.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
No log	0.2183	200	0.1927	0.4257
No log	0.4367	400	0.1713	0.3885
2.0358	0.6550	600	0.1760	0.3907
2.0358	0.8734	800	0.1819	0.4143
0.519	1.0917	1000	0.1611	0.3869
0.519	1.3100	1200	0.1550	0.3736
0.519	1.5284	1400	0.1538	0.3771
0.4764	1.7467	1600	0.1744	0.4176
0.4764	1.9651	1800	0.1598	0.3884
0.4501	2.1834	2000	0.1507	0.3577
0.4501	2.4017	2200	0.1535	0.3763
0.4501	2.6201	2400	0.1502	0.3649
0.4422	2.8384	2600	0.1457	0.3502
0.4422	3.0568	2800	0.1485	0.3580
0.4217	3.2751	3000	0.1480	0.3547
0.4217	3.4934	3200	0.1498	0.3666
0.4217	3.7118	3400	0.1458	0.3494
0.4144	3.9301	3600	0.1427	0.3574
0.4144	4.1485	3800	0.1445	0.3594
0.3926	4.3668	4000	0.1462	0.3666
0.3926	4.5852	4200	0.1432	0.3527
0.3926	4.8035	4400	0.1409	0.3498

Framework versions

• Transformers 4.43.0
• Pytorch 2.3.1+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1