mms-1b-all-bem-genbed-f-model

This model is a fine-tuned version of facebook/mms-1b-all on the GENBED - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.1823
• Wer: 0.3431

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
6.6556	0.1370	100	0.5951	0.6643
0.4415	0.2740	200	0.2734	0.4565
0.3448	0.4110	300	0.2482	0.4289
0.3459	0.5479	400	0.2392	0.4149
0.3184	0.6849	500	0.2304	0.4085
0.3058	0.8219	600	0.2372	0.4108
0.3077	0.9589	700	0.2271	0.4172
0.2812	1.0959	800	0.2217	0.3983
0.3297	1.2329	900	0.2209	0.3984
0.2817	1.3699	1000	0.2163	0.4124
0.2927	1.5068	1100	0.2146	0.3863
0.2806	1.6438	1200	0.2106	0.3851
0.2574	1.7808	1300	0.2098	0.3866
0.2829	1.9178	1400	0.2067	0.3772
0.2764	2.0548	1500	0.2076	0.3789
0.2635	2.1918	1600	0.2076	0.3769
0.2761	2.3288	1700	0.2068	0.3801
0.2854	2.4658	1800	0.1994	0.3645
0.2557	2.6027	1900	0.2016	0.3861
0.2717	2.7397	2000	0.2011	0.3734
0.2504	2.8767	2100	0.1989	0.3674
0.2606	3.0137	2200	0.1990	0.3835
0.2583	3.1507	2300	0.2028	0.3666
0.2591	3.2877	2400	0.1952	0.3507
0.2408	3.4247	2500	0.1988	0.3637
0.2485	3.5616	2600	0.1972	0.3593
0.2474	3.6986	2700	0.1949	0.3534
0.2398	3.8356	2800	0.1959	0.3697
0.2512	3.9726	2900	0.1906	0.3559
0.2266	4.1096	3000	0.1905	0.3482
0.2538	4.2466	3100	0.1916	0.3521
0.2268	4.3836	3200	0.1914	0.3895
0.2249	4.5205	3300	0.1897	0.3417
0.2416	4.6575	3400	0.1877	0.3458
0.2421	4.7945	3500	0.1872	0.3412
0.244	4.9315	3600	0.1855	0.3528
0.2371	5.0685	3700	0.1871	0.3447
0.2383	5.2055	3800	0.1833	0.3523
0.2409	5.3425	3900	0.1886	0.3487
0.2312	5.4795	4000	0.1848	0.3438
0.2261	5.6164	4100	0.1866	0.3469
0.2169	5.7534	4200	0.1841	0.3376
0.2283	5.8904	4300	0.1865	0.3412
0.2182	6.0274	4400	0.1823	0.3431
0.2141	6.1644	4500	0.1858	0.3403
0.2127	6.3014	4600	0.1876	0.3356
0.229	6.4384	4700	0.1863	0.3361

Framework versions

• Transformers 4.46.0.dev0
• Pytorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.20.0