asr-africa/w2v-bert-2.0-yogera-lg-1hr-v2

wav2vec2-bert

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the Yogera dataset. It achieves the following results on the evaluation set:

• Loss: 0.5427
• Wer: 0.2853
• Cer: 0.0569

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
4.9747	1.0	12	3.1195	1.0	0.9999
2.8001	2.0	24	2.1571	1.0	0.7639
1.428	3.0	36	0.7543	0.9173	0.2242
0.7032	4.0	48	0.4636	0.5435	0.1101
0.4798	5.0	60	0.3611	0.4315	0.0871
0.2475	6.0	72	0.3558	0.3894	0.0798
0.1798	7.0	84	0.3404	0.3351	0.0684
0.1978	8.0	96	0.3484	0.3530	0.0711
0.1186	9.0	108	0.3509	0.3305	0.0662
0.0942	10.0	120	0.3617	0.3093	0.0648
0.081	11.0	132	0.3550	0.3110	0.0630
0.0627	12.0	144	0.3864	0.3035	0.0610
0.0523	13.0	156	0.3893	0.2903	0.0584
0.0688	14.0	168	0.4161	0.3142	0.0625
0.0695	15.0	180	0.3915	0.2913	0.0593
0.0331	16.0	192	0.4062	0.2893	0.0595
0.0241	17.0	204	0.3998	0.2865	0.0580
0.0205	18.0	216	0.4433	0.2877	0.0600
0.0298	19.0	228	0.4508	0.2980	0.0600
0.023	20.0	240	0.4631	0.2961	0.0612
0.0451	21.0	252	0.4295	0.3091	0.0617
0.0252	22.0	264	0.4577	0.2895	0.0592
0.0163	23.0	276	0.4462	0.2866	0.0586
0.0143	24.0	288	0.4808	0.2938	0.0591
0.0099	25.0	300	0.4952	0.2883	0.0583
0.0088	26.0	312	0.4998	0.2795	0.0581
0.0076	27.0	324	0.5088	0.2886	0.0583
0.0054	28.0	336	0.5035	0.2755	0.0561
0.0033	29.0	348	0.5032	0.2706	0.0555
0.0025	30.0	360	0.5087	0.2674	0.0546
0.0024	31.0	372	0.5132	0.2684	0.0543
0.0031	32.0	384	0.5107	0.2679	0.0543
0.0036	33.0	396	0.5014	0.2687	0.0550
0.0068	34.0	408	0.5016	0.2691	0.0550
0.0019	35.0	420	0.5064	0.2722	0.0561
0.0009	36.0	432	0.5101	0.2704	0.0558
0.0018	37.0	444	0.5331	0.2701	0.0552
0.0009	38.0	456	0.5290	0.2691	0.0558
0.0009	39.0	468	0.5267	0.2711	0.0561
0.0035	40.0	480	0.5404	0.2791	0.0575
0.0027	41.0	492	0.5386	0.2895	0.0576
0.004	42.0	504	0.5436	0.2815	0.0571
0.0042	43.0	516	0.5404	0.2858	0.0583
0.0035	44.0	528	0.5519	0.2797	0.0570
0.004	45.0	540	0.5427	0.2853	0.0569

Framework versions

• Transformers 4.45.2
• Pytorch 2.1.0+cu118
• Datasets 3.0.1
• Tokenizers 0.20.1