---
license: mit
base_model: facebook/w2v-bert-2.0
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: w2v-bert-2.0-tamil-gpu-custom_preprocessed_v2
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# w2v-bert-2.0-tamil-gpu-custom_preprocessed_v2

This model is a fine-tuned version of [facebook/w2v-bert-2.0](https://huggingface.co/facebook/w2v-bert-2.0) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: inf
- Wer: 0.4310

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 4.53567e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 10
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Wer    |
|:-------------:|:-----:|:-----:|:---------------:|:------:|
| 3.2473        | 0.24  | 300   | inf             | 0.4771 |
| 0.7121        | 0.49  | 600   | inf             | 0.3487 |
| 0.552         | 0.73  | 900   | inf             | 0.3140 |
| 0.4973        | 0.97  | 1200  | inf             | 0.3202 |
| 0.499         | 1.22  | 1500  | inf             | 0.2678 |
| 0.4667        | 1.46  | 1800  | inf             | 0.2784 |
| 0.5909        | 1.71  | 2100  | inf             | 0.3930 |
| 1.411         | 1.95  | 2400  | inf             | 0.3839 |
| 2.1124        | 2.19  | 2700  | inf             | 0.4063 |
| 2.2398        | 2.44  | 3000  | inf             | 0.4310 |
| 2.3058        | 2.68  | 3300  | inf             | 0.4310 |
| 2.262         | 2.92  | 3600  | inf             | 0.4310 |
| 2.2588        | 3.17  | 3900  | inf             | 0.4310 |
| 2.3649        | 3.41  | 4200  | inf             | 0.4310 |
| 2.2835        | 3.66  | 4500  | inf             | 0.4310 |
| 2.3228        | 3.9   | 4800  | inf             | 0.4310 |
| 2.2322        | 4.14  | 5100  | inf             | 0.4310 |
| 2.3131        | 4.39  | 5400  | inf             | 0.4310 |
| 2.2916        | 4.63  | 5700  | inf             | 0.4310 |
| 2.3239        | 4.87  | 6000  | inf             | 0.4310 |
| 2.3533        | 5.12  | 6300  | inf             | 0.4310 |
| 2.2787        | 5.36  | 6600  | inf             | 0.4310 |
| 2.2776        | 5.61  | 6900  | inf             | 0.4310 |
| 2.3143        | 5.85  | 7200  | inf             | 0.4310 |
| 2.3105        | 6.09  | 7500  | inf             | 0.4310 |
| 2.2639        | 6.34  | 7800  | inf             | 0.4310 |
| 2.3211        | 6.58  | 8100  | inf             | 0.4310 |
| 2.2755        | 6.82  | 8400  | inf             | 0.4310 |
| 2.3074        | 7.07  | 8700  | inf             | 0.4310 |
| 2.2627        | 7.31  | 9000  | inf             | 0.4310 |
| 2.2756        | 7.55  | 9300  | inf             | 0.4310 |
| 2.2594        | 7.8   | 9600  | inf             | 0.4310 |
| 2.2221        | 8.04  | 9900  | inf             | 0.4310 |
| 2.2932        | 8.29  | 10200 | inf             | 0.4310 |
| 2.2978        | 8.53  | 10500 | inf             | 0.4310 |
| 2.2958        | 8.77  | 10800 | inf             | 0.4310 |
| 2.3239        | 9.02  | 11100 | inf             | 0.4310 |
| 2.281         | 9.26  | 11400 | inf             | 0.4310 |
| 2.272         | 9.5   | 11700 | inf             | 0.4310 |
| 2.2544        | 9.75  | 12000 | inf             | 0.4310 |
| 2.3103        | 9.99  | 12300 | inf             | 0.4310 |


### Framework versions

- Transformers 4.39.3
- Pytorch 2.1.2+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2