---
license: mit
base_model: facebook/w2v-bert-2.0
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: w2v-bert-2.0-ln-afrivoice-10hr-v1
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# w2v-bert-2.0-ln-afrivoice-10hr-v1

This model is a fine-tuned version of [facebook/w2v-bert-2.0](https://huggingface.co/facebook/w2v-bert-2.0) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3773
- Model Preparation Time: 0.0146
- Wer: 0.2579
- Cer: 0.0680

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 32
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.05
- num_epochs: 100
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Model Preparation Time | Wer    | Cer    |
|:-------------:|:-----:|:----:|:---------------:|:----------------------:|:------:|:------:|
| 5.4983        | 1.0   | 31   | 2.8460          | 0.0146                 | 1.0    | 1.0000 |
| 2.3085        | 2.0   | 62   | 1.1224          | 0.0146                 | 0.7398 | 0.2589 |
| 0.8931        | 3.0   | 93   | 0.8311          | 0.0146                 | 0.4138 | 0.1536 |
| 0.7396        | 4.0   | 124  | 0.7901          | 0.0146                 | 0.7207 | 0.2228 |
| 0.6648        | 5.0   | 155  | 0.7516          | 0.0146                 | 0.4619 | 0.1744 |
| 0.6398        | 6.0   | 186  | 0.7282          | 0.0146                 | 0.4477 | 0.1611 |
| 0.586         | 7.0   | 217  | 0.8923          | 0.0146                 | 0.3608 | 0.1344 |
| 0.5666        | 8.0   | 248  | 0.7070          | 0.0146                 | 0.4131 | 0.1518 |
| 0.4841        | 9.0   | 279  | 0.7248          | 0.0146                 | 0.3747 | 0.1437 |
| 0.4442        | 10.0  | 310  | 0.8866          | 0.0146                 | 0.3733 | 0.1370 |
| 0.4325        | 11.0  | 341  | 0.6852          | 0.0146                 | 0.3717 | 0.1429 |
| 0.3584        | 12.0  | 372  | 0.8831          | 0.0146                 | 0.3453 | 0.1300 |
| 0.3397        | 13.0  | 403  | 0.9853          | 0.0146                 | 0.3546 | 0.1314 |
| 0.3053        | 14.0  | 434  | 0.7321          | 0.0146                 | 0.3924 | 0.1509 |
| 0.2655        | 15.0  | 465  | 0.8055          | 0.0146                 | 0.3672 | 0.1386 |
| 0.2555        | 16.0  | 496  | 0.8419          | 0.0146                 | 0.3725 | 0.1405 |
| 0.2239        | 17.0  | 527  | 0.8440          | 0.0146                 | 0.3850 | 0.1449 |
| 0.1853        | 18.0  | 558  | 0.9243          | 0.0146                 | 0.3681 | 0.1397 |
| 0.1554        | 19.0  | 589  | 0.9458          | 0.0146                 | 0.3835 | 0.1452 |
| 0.125         | 20.0  | 620  | 1.2681          | 0.0146                 | 0.3506 | 0.1309 |
| 0.1053        | 21.0  | 651  | 1.3723          | 0.0146                 | 0.3682 | 0.1324 |
| 0.0776        | 22.0  | 682  | 1.3026          | 0.0146                 | 0.3781 | 0.1369 |


### Framework versions

- Transformers 4.44.0
- Pytorch 2.1.0+cu118
- Datasets 2.20.0
- Tokenizers 0.19.1