fine-tuned-QAS-Squad_2-with-roberta-large

This model is a fine-tuned version of roberta-large on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7481
• Exact Match: 71.7912
• F1: 85.1553

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1
0.957	0.5	463	0.8279	64.8987	79.7543
0.7977	1.0	926	0.7340	68.8325	82.9258
0.6992	1.5	1389	0.7095	69.8327	83.3647
0.6556	2.0	1852	0.6849	70.2278	84.0408
0.5743	2.5	2315	0.6992	70.4715	84.3736
0.574	3.0	2778	0.6917	70.9507	85.1835
0.4734	3.5	3241	0.7291	70.8330	84.8717
0.4733	4.0	3704	0.6828	71.6567	85.1160
0.4171	4.5	4167	0.7481	71.7912	85.1553

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu117
• Datasets 2.2.0
• Tokenizers 0.13.2