metadata

license: apache-2.0
base_model: facebook/wav2vec2-large-xlsr-53
tags:
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: wav2vec2-xlsr-53-ft-btb-cy
    results: []

wav2vec2-xlsr-53-ft-btb-cy

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.4105
Wer: 0.3136

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0003
train_batch_size: 16
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 500
training_steps: 2600
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
No log	0.1414	100	3.2195	1.0
No log	0.2829	200	3.1180	1.0
No log	0.4243	300	1.7236	0.9364
No log	0.5658	400	1.1579	0.7628
3.0279	0.7072	500	0.9558	0.6721
3.0279	0.8487	600	0.8189	0.6424
3.0279	0.9901	700	0.6973	0.5164
3.0279	1.1315	800	0.6183	0.4752
3.0279	1.2730	900	0.5936	0.4703
0.7925	1.4144	1000	0.5498	0.4304
0.7925	1.5559	1100	0.5286	0.4148
0.7925	1.6973	1200	0.5130	0.3974
0.7925	1.8388	1300	0.4878	0.3864
0.7925	1.9802	1400	0.4740	0.3733
0.62	2.1216	1500	0.4578	0.3593
0.62	2.2631	1600	0.4535	0.3500
0.62	2.4045	1700	0.4485	0.3497
0.62	2.5460	1800	0.4373	0.3431
0.62	2.6874	1900	0.4362	0.3429
0.4879	2.8289	2000	0.4236	0.3327
0.4879	2.9703	2100	0.4172	0.3257
0.4879	3.1117	2200	0.4206	0.3217
0.4879	3.2532	2300	0.4166	0.3199
0.4879	3.3946	2400	0.4134	0.3173
0.4036	3.5361	2500	0.4110	0.3159
0.4036	3.6775	2600	0.4105	0.3136

Framework versions

Transformers 4.40.2
Pytorch 2.3.0+cu121
Datasets 2.19.1
Tokenizers 0.19.1