okezieowen/whisper-small-naija-9-19-2024

whisper-small-multilingual-naija-9-19-2024

This model is a fine-tuned version of openai/whisper-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3460
• Wer: 26.4719

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 3.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
1.0057	0.0997	250	1.0392	143.2330
0.7769	0.1994	500	0.8744	87.4356
0.6718	0.2992	750	0.7590	59.3527
0.5808	0.3989	1000	0.7119	59.9959
0.5277	0.4986	1250	0.6492	51.7774
0.4905	0.5983	1500	0.6179	51.2283
0.469	0.6980	1750	0.5779	47.8888
0.4354	0.7978	2000	0.5559	47.9575
0.4193	0.8975	2250	0.5324	46.0621
0.3942	0.9972	2500	0.5135	48.0926
0.2432	1.0969	2750	0.5089	46.1072
0.2417	1.1966	3000	0.4927	43.0126
0.2237	1.2964	3250	0.4829	45.9237
0.2591	1.3961	3500	0.4626	40.8138
0.2037	1.4958	3750	0.4468	39.4518
0.2136	1.5955	4000	0.4360	37.9211
0.2212	1.6953	4250	0.4207	37.9017
0.2033	1.7950	4500	0.4092	36.7122
0.1947	1.8947	4750	0.4001	38.3042
0.1617	1.9944	5000	0.3855	33.8600
0.0718	2.0941	5250	0.3893	34.5793
0.0703	2.1939	5500	0.3839	32.1661
0.0664	2.2936	5750	0.3791	32.3118
0.0729	2.3933	6000	0.3743	31.5436
0.0633	2.4930	6250	0.3684	30.5690
0.0568	2.5927	6500	0.3616	28.4227
0.0573	2.6925	6750	0.3555	28.2268
0.0643	2.7922	7000	0.3526	28.1880
0.0659	2.8919	7250	0.3479	26.6771
0.0532	2.9916	7500	0.3460	26.4719

Framework versions

• Transformers 4.44.2
• Pytorch 2.1.0+cu118
• Datasets 3.0.0
• Tokenizers 0.19.1