SHK_Urdu_Fine_Tuned_2

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the common_voice_8_0 dataset. It achieves the following results on the evaluation set:

• Loss: 1.3107
• Wer: 0.7347

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
16.3705	1.69	100	4.1962	1.0
3.415	3.39	200	3.2513	1.0
3.2396	5.08	300	3.2481	1.0
3.2109	6.78	400	3.1929	1.0
3.1661	8.47	500	3.1312	1.0
3.1014	10.17	600	3.1059	0.9993
3.0761	11.86	700	3.0239	1.0
2.9687	13.56	800	2.8371	1.0
2.7406	15.25	900	2.4320	0.9978
2.2203	16.95	1000	2.0063	0.9834
1.7338	18.64	1100	1.6290	0.8753
1.3901	20.34	1200	1.4847	0.7904
1.1757	22.03	1300	1.3944	0.7649
1.0581	23.73	1400	1.3455	0.7520
0.9609	25.42	1500	1.3346	0.7417
0.9147	27.12	1600	1.3014	0.7339
0.8662	28.81	1700	1.3107	0.7347

Framework versions

• Transformers 4.21.0
• Pytorch 1.11.0+cu113
• Datasets 2.4.0
• Tokenizers 0.12.1