facebook/wav2vec2-large-xlsr-53 · The hardware requirement for above model for training

Aug 19

Dear Team,

I am using wav2vec2-large-xlsr-53model for training the Indian Languages.
When I train on 6 GB GPU RAM, it gives me out of memory error.

Please let me know what is the minimum GPU-RAM required for training the model.

Anil

nyarkssss

Aug 21

Hello, have you found an answer to this question?

Daedalean

Sep 20

•

edited Sep 20

Fine-tuning Wav2Vec2-XLSR-53 on a custom Japanese dataset with the following training arguments required a little less than 80 GB of GPU RAM for my use case, if I am understanding vast.ai's metrics correctly.

training_args = TrainingArguments(
    output_dir="facebook-wav2vec2-large-xlsr-53-ja",
    group_by_length=False,
    per_device_train_batch_size=32,
    evaluation_strategy="steps",
    max_steps=655000,
    fp16=False,
    gradient_checkpointing=True,
    save_steps=500,
    eval_steps=500,
    logging_steps=500,
    learning_rate=3e-4,
    warmup_steps=1000,
    save_total_limit=2,
)

facebook
/

wav2vec2-large-xlsr-53

The hardware requirement for above model for training - GPU RAM size