Hugging Face's logo

DewiBrynJones
/
w2v2-bert-ft-btb-cy

Automatic Speech Recognition
Transformers
TensorBoard
Safetensors
wav2vec2-bert
DewiBrynJones/banc-trawsgrifiadau-bangor-normalized
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
w2v2-bert-ft-btb-cy / README.md
DewiBrynJones's picture
DewiBrynJones
End of training
5dae949 verified
preview code
|
raw
history blame
3.02 kB
metadata
license: mit
base_model: facebook/w2v-bert-2.0
tags:
  - automatic-speech-recognition
  - DewiBrynJones/banc-trawsgrifiadau-bangor-normalized
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: w2v2-bert-ft-btb-cy
    results: []

w2v2-bert-ft-btb-cy

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the DEWIBRYNJONES/BANC-TRAWSGRIFIADAU-BANGOR-NORMALIZED - DEFAULT dataset. It achieves the following results on the evaluation set:

  • Loss: 2.9375
  • Wer: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 500
  • training_steps: 2500
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
No log 0.1414 100 6.9702 1.0
No log 0.2829 200 3.3676 1.0
No log 0.4243 300 3.0393 1.0
No log 0.5658 400 2.9887 1.0
4.717 0.7072 500 3.0227 1.0
4.717 0.8487 600 3.0406 1.0
4.717 0.9901 700 3.0029 1.0
4.717 1.1315 800 2.9483 1.0
4.717 1.2730 900 2.9511 1.0
3.0065 1.4144 1000 2.9473 1.0
3.0065 1.5559 1100 2.9448 1.0
3.0065 1.6973 1200 2.9470 1.0
3.0065 1.8388 1300 2.9446 1.0
3.0065 1.9802 1400 2.9432 1.0
2.9634 2.1216 1500 2.9476 1.0
2.9634 2.2631 1600 2.9624 1.0
2.9634 2.4045 1700 2.9581 1.0
2.9634 2.5460 1800 2.9553 1.0
2.9634 2.6874 1900 2.9515 1.0
2.9677 2.8289 2000 2.9481 1.0
2.9677 2.9703 2100 2.9509 1.0
2.9677 3.1117 2200 2.9408 1.0
2.9677 3.2532 2300 2.9393 1.0
2.9677 3.3946 2400 2.9381 1.0
2.9612 3.5361 2500 2.9375 1.0

Framework versions

  • Transformers 4.40.2
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.1
  • Tokenizers 0.19.1