Hugging Face's logo

gttsehu
/
wav2vec2-xls-r-300m-bp1-es_eu

Automatic Speech Recognition
Transformers
Safetensors
wav2vec2
gttsehu/basque_parliament_1
Generated from Trainer
Inference Endpoints
Model card Files Community
wav2vec2-xls-r-300m-bp1-es_eu / README.md
mpenagar's picture
mpenagar
Initial release
177a846
preview code
|
raw
history blame
3.88 kB
metadata
license: apache-2.0
base_model: facebook/wav2vec2-xls-r-300m
tags:
  - automatic-speech-recognition
  - gttsehu/basque_parliament_1
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: facebook/wav2vec2-xls-r-300m
    results: []

facebook/wav2vec2-xls-r-300m

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the GTTSEHU/BASQUE_PARLIAMENT_1 - NA dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0846
  • Wer: 0.0367
  • Cer: 0.0132

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 4
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 32
  • total_eval_batch_size: 16
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 1000
  • num_epochs: 6.0
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Cer
0.7054 0.19 4000 0.1011 0.0871 0.0227
0.0856 0.39 8000 0.0995 0.0747 0.0207
0.075 0.58 12000 0.0868 0.0647 0.0185
0.0694 0.77 16000 0.0853 0.0619 0.0183
0.0658 0.97 20000 0.0778 0.0573 0.0171
0.0589 1.16 24000 0.0821 0.0546 0.0166
0.0572 1.35 28000 0.0827 0.0558 0.0170
0.0551 1.55 32000 0.0830 0.0533 0.0169
0.054 1.74 36000 0.0788 0.0512 0.0162
0.0524 1.93 40000 0.0783 0.0489 0.0156
0.048 2.13 44000 0.0861 0.0492 0.0160
0.046 2.32 48000 0.0763 0.0494 0.0154
0.0456 2.51 52000 0.0835 0.0471 0.0153
0.0439 2.71 56000 0.0790 0.0469 0.0152
0.0436 2.9 60000 0.0832 0.0472 0.0155
0.0406 3.09 64000 0.0810 0.0442 0.0148
0.0386 3.29 68000 0.0810 0.0436 0.0146
0.038 3.48 72000 0.0778 0.0430 0.0143
0.0373 3.67 76000 0.0785 0.0430 0.0144
0.0363 3.87 80000 0.0788 0.0421 0.0144
0.0348 4.06 84000 0.0823 0.0423 0.0144
0.0323 4.25 88000 0.0819 0.0407 0.0143
0.0319 4.45 92000 0.0809 0.0410 0.0142
0.0314 4.64 96000 0.0821 0.0400 0.0138
0.0306 4.83 100000 0.0813 0.0389 0.0137
0.0295 5.03 104000 0.0820 0.0377 0.0131
0.0275 5.22 108000 0.0866 0.0378 0.0137
0.0267 5.41 112000 0.0831 0.0376 0.0134
0.0264 5.61 116000 0.0845 0.0369 0.0132
0.0258 5.8 120000 0.0859 0.0370 0.0133
0.0254 6.0 124000 0.0846 0.0367 0.0132

Framework versions

  • Transformers 4.36.0.dev0
  • Pytorch 2.1.1+cu121
  • Datasets 2.15.1.dev0
  • Tokenizers 0.15.0