bert-27

This model is a fine-tuned version of deepset/bert-base-cased-squad2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 12.0964

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
11.2271	0.18	5	12.3380
10.9992	0.36	10	12.3098
11.1726	0.54	15	12.2822
10.9587	0.71	20	12.2569
10.864	0.89	25	12.2331
10.7044	1.07	30	12.2122
10.9072	1.25	35	12.1926
10.7844	1.43	40	12.1746
10.7675	1.61	45	12.1583
10.8238	1.79	50	12.1439
10.9665	1.96	55	12.1312
10.7927	2.14	60	12.1204
10.5926	2.32	65	12.1109
10.6499	2.5	70	12.1043
11.0197	2.68	75	12.0999
10.6181	2.86	80	12.0964

Framework versions

• Transformers 4.34.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.14.1