mms-zeroshot-bem-sv-male

This model is a fine-tuned version of mms-meta/mms-zeroshot-300m on the BEMBASPEECH - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.1874
• Wer: 0.3949

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 10.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
No log	0.2183	200	2.3822	1.0
No log	0.4367	400	0.2715	0.5093
2.7769	0.6550	600	0.2489	0.4820
2.7769	0.8734	800	0.2296	0.4695
0.6809	1.0917	1000	0.2209	0.4638
0.6809	1.3100	1200	0.2163	0.4469
0.6809	1.5284	1400	0.2092	0.4400
0.6113	1.7467	1600	0.2047	0.4346
0.6113	1.9651	1800	0.2074	0.4467
0.5974	2.1834	2000	0.2041	0.4304
0.5974	2.4017	2200	0.2054	0.4317
0.5974	2.6201	2400	0.1987	0.4240
0.5636	2.8384	2600	0.2003	0.4252
0.5636	3.0568	2800	0.1997	0.4287
0.5398	3.2751	3000	0.2097	0.4400
0.5398	3.4934	3200	0.1968	0.4165
0.5398	3.7118	3400	0.2013	0.4218
0.5334	3.9301	3600	0.2003	0.4230
0.5334	4.1485	3800	0.1976	0.4227
0.5123	4.3668	4000	0.1978	0.4198
0.5123	4.5852	4200	0.2019	0.4298
0.5123	4.8035	4400	0.1939	0.4146
0.5119	5.0218	4600	0.1989	0.4161
0.5119	5.2402	4800	0.1902	0.4076
0.4929	5.4585	5000	0.1929	0.4116
0.4929	5.6769	5200	0.1943	0.4144
0.4929	5.8952	5400	0.1922	0.4106
0.4878	6.1135	5600	0.1933	0.4137
0.4878	6.3319	5800	0.1920	0.4058
0.4755	6.5502	6000	0.1927	0.4171
0.4755	6.7686	6200	0.1920	0.4127
0.4755	6.9869	6400	0.1925	0.4061
0.475	7.2052	6600	0.1884	0.4058
0.475	7.4236	6800	0.1903	0.4070
0.4715	7.6419	7000	0.1882	0.3996
0.4715	7.8603	7200	0.1881	0.4033
0.4715	8.0786	7400	0.1885	0.4007
0.4575	8.2969	7600	0.1885	0.4016
0.4575	8.5153	7800	0.1888	0.4050
0.4611	8.7336	8000	0.1884	0.4046
0.4611	8.9520	8200	0.1881	0.3974
0.4611	9.1703	8400	0.1865	0.3956
0.4559	9.3886	8600	0.1875	0.3974
0.4559	9.6070	8800	0.1872	0.3996
0.4536	9.8253	9000	0.1876	0.3953

Framework versions

• Transformers 4.45.0.dev0
• Pytorch 2.4.0+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1