nb-whisper-small-karelian-CodeSwitching

This model is a fine-tuned version of NbAiLab/nb-whisper-small-verbatim on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7241
• Wer: 0.3278
• Cer: 0.0999

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• distributed_type: multi-GPU
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 10000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.1848	1.1338	500	0.6758	0.4066	0.1227
0.0796	2.2676	1000	0.6685	0.3549	0.1015
0.0492	3.4014	1500	0.7090	0.3758	0.1088
0.0426	4.5351	2000	0.7095	0.3704	0.1083
0.0311	5.6689	2500	0.7148	0.3742	0.1138
0.027	6.8027	3000	0.7173	0.3454	0.1004
0.0201	7.9365	3500	0.7261	0.3813	0.1325
0.0165	9.0703	4000	0.7158	0.3417	0.0982
0.0179	10.2041	4500	0.7261	0.3495	0.1036
0.0101	11.3379	5000	0.7275	0.3315	0.0978
0.0086	12.4717	5500	0.7437	0.3400	0.1081
0.0058	13.6054	6000	0.7524	0.3410	0.1026
0.0056	14.7392	6500	0.7256	0.3407	0.1015
0.0075	15.8730	7000	0.7202	0.3312	0.0987
0.0047	17.0068	7500	0.7266	0.3359	0.1025
0.0046	18.1406	8000	0.7271	0.3312	0.0973
0.0039	19.2744	8500	0.7334	0.3353	0.0999
0.0025	20.4082	9000	0.7280	0.3295	0.0987
0.0022	21.5420	9500	0.7290	0.3254	0.0972
0.0031	22.6757	10000	0.7241	0.3278	0.0999

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1