Edit model card

Wav2Vec2 XLS-R Adult/Child Indonesian Speech Classifier

Wav2Vec2 XLS-R Adult/Child Indonesian Speech Classifier is an audio classification model based on the XLS-R architecture. This model is a fine-tuned version of wav2vec2-xls-r-300m on a private adult/child Indonesian speech classification dataset.

This model was trained using HuggingFace's PyTorch framework. All training was done on a Tesla P100, provided by Kaggle. Training metrics were logged via Tensorboard.

Model

Model #params Arch. Training/Validation data (text)
wav2vec2-xls-r-adult-child-id-cls 300M XLS-R Adult/Child Indonesian Speech Classification Dataset

Evaluation Results

The model achieves the following results on evaluation:

Dataset Loss Accuracy F1
Adult/Child Indonesian Speech Classification 0.1970 93.38% 0.9307

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 3e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 4

Training results

Training Loss Epoch Step Validation Loss Accuracy F1
0.336 1.0 305 0.3146 0.8845 0.8698
0.2345 2.0 610 0.2140 0.9251 0.9202
0.3215 3.0 915 0.2038 0.9315 0.9286
0.2059 4.0 1220 0.1970 0.9338 0.9307

Disclaimer

Do consider the biases which came from pre-training datasets that may be carried over into the results of this model.

Authors

Wav2Vec2 XLS-R Adult/Child Indonesian Speech Classifier was trained and evaluated by Ananto Joyoadikusumo. All computation and development are done on Kaggle.

Framework versions

  • Transformers 4.18.0
  • Pytorch 1.11.0+cu102
  • Datasets 2.2.0
  • Tokenizers 0.12.1
Downloads last month
10
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.