zuazo/whisper-tiny-eu-cv16_1

Whisper Tiny Basque

This model is a fine-tuned version of openai/whisper-tiny on the mozilla-foundation/common_voice_16_1 eu dataset. It achieves the following results on the evaluation set:

• Loss: 0.5146
• Wer: 19.0949

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3.75e-05
• train_batch_size: 256
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 40000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.0426	10.0	1000	0.3451	23.2003
0.0077	20.0	2000	0.4123	22.6053
0.0013	30.0	3000	0.4288	21.1965
0.0004	40.0	4000	0.4538	21.1926
0.0003	50.0	5000	0.4757	21.1808
0.0206	60.0	6000	0.4172	22.2751
0.0003	70.0	7000	0.4374	19.5131
0.0002	80.0	8000	0.4547	19.5091
0.0001	90.0	9000	0.4697	19.5062
0.0001	100.0	10000	0.4853	19.5199
0.0001	110.0	11000	0.5009	19.5687
0.0	120.0	12000	0.5175	19.6586
0.0	130.0	13000	0.5348	19.7729
0.0	140.0	14000	0.5531	19.7847
0.0002	150.0	15000	0.4626	19.4730
0.0001	160.0	16000	0.4813	19.2199
0.0	170.0	17000	0.4932	19.1691
0.0	180.0	18000	0.5041	19.1291
0.0	190.0	19000	0.5146	19.0949
0.0	200.0	20000	0.5254	19.1232
0.0	210.0	21000	0.5369	19.1369
0.0	220.0	22000	0.5484	19.1125
0.0	230.0	23000	0.5606	19.1330
0.0	240.0	24000	0.5732	19.1965
0.0	250.0	25000	0.5864	19.2219
0.0	260.0	26000	0.6003	19.3108
0.0	270.0	27000	0.6140	19.3714
0.0034	280.0	28000	0.5536	20.6630
0.0	290.0	29000	0.5486	19.3391
0.0	300.0	30000	0.5591	19.3059
0.0	310.0	31000	0.5669	19.3137
0.0	320.0	32000	0.5737	19.3225
0.0	330.0	33000	0.5798	19.2883
0.0	340.0	34000	0.5856	19.2668
0.0	350.0	35000	0.5911	19.2346
0.0	360.0	36000	0.5962	19.2287
0.0	370.0	37000	0.6010	19.2326
0.0	380.0	38000	0.6050	19.2287
0.0	390.0	39000	0.6081	19.2375
0.0	400.0	40000	0.6095	19.1965

Framework versions

• Transformers 4.37.2
• Pytorch 2.2.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.1