Hugging Face's logo

LevonHakobyan
/
adapter_freezed_base_const_lr

Automatic Speech Recognition
Transformers
TensorBoard
Safetensors
wav2vec2-bert
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Metrics Community
adapter_freezed_base_const_lr / README.md
LevonHakobyan's picture
LevonHakobyan
End of training
7c6cf8d verified
preview code
|
raw
history blame
2.91 kB
metadata
license: mit
base_model: facebook/w2v-bert-2.0
tags:
  - generated_from_trainer
datasets:
  - common_voice_17_0
metrics:
  - wer
model-index:
  - name: adapter_freezed_base_const_lr
    results:
      - task:
          name: Automatic Speech Recognition
          type: automatic-speech-recognition
        dataset:
          name: common_voice_17_0
          type: common_voice_17_0
          config: hy-AM
          split: test
          args: hy-AM
        metrics:
          - name: Wer
            type: wer
            value: 0.9281584969288209

adapter_freezed_base_const_lr

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the common_voice_17_0 dataset. It achieves the following results on the evaluation set:

  • Loss: 0.9200
  • Wer: 0.9282
  • Cer: 0.2562

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: constant
  • num_epochs: 10
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Cer
1.3224 0.6154 200 1.3171 0.9949 0.3890
1.02 1.2308 400 1.0780 0.9728 0.3233
0.9256 1.8462 600 0.9799 0.9738 0.2955
0.8377 2.4615 800 0.9756 0.9663 0.2919
0.7836 3.0769 1000 0.9143 0.9535 0.2730
0.7516 3.6923 1200 0.8908 0.9373 0.2671
0.6714 4.3077 1400 0.9088 0.9497 0.2692
0.6749 4.9231 1600 0.9006 0.9566 0.2681
0.6223 5.5385 1800 0.8686 0.9322 0.2587
0.5643 6.1538 2000 0.8846 0.9422 0.2580
0.5773 6.7692 2200 0.8960 0.9396 0.2644
0.5067 7.3846 2400 0.8778 0.9273 0.2545
0.5123 8.0 2600 0.8919 0.9379 0.2601
0.4729 8.6154 2800 0.9131 0.9597 0.2587
0.406 9.2308 3000 0.9032 0.9389 0.2564
0.4286 9.8462 3200 0.9200 0.9282 0.2562

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.3.0+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1