deberta-v3-large-AVeriTeC-nli

This model was finetuned from microsoft/deberta-v3-large on an AVeriTec dataset. It achieves the following results on the evaluation set:

Intended uses & limitations

This model is intended for usage in a pipeline for open-domain fact-checking task.

Training and evaluation data

See chenxwh/AVeriTeC

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: adamw_torch
• training_precision: float16
• learning_rate: 1e-5
• per_device_train_batch_size: 32
• num_train_epochs: 10
• weight_decay: 0.01
• load_best_model_at_end: True #early stopping!
• warmup_ratio: 0.06

Training results

Framework versions

• Transformers 4.43.0
• TensorFlow 2.17.0
• Datasets 2.20.0
• Tokenizers 0.19.1