distilroberta-base-DoniaTrials514

This model is a fine-tuned version of distilroberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.5493

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	39	3.2269
No log	2.0	78	3.1520
No log	3.0	117	3.0998
No log	4.0	156	3.0586
No log	5.0	195	3.0225
No log	6.0	234	2.9860
No log	7.0	273	2.9566
No log	8.0	312	2.9297
No log	9.0	351	2.9042
No log	10.0	390	2.8782
No log	11.0	429	2.8561
No log	12.0	468	2.8347
3.0074	13.0	507	2.8161
3.0074	14.0	546	2.8003
3.0074	15.0	585	2.7840
3.0074	16.0	624	2.7666
3.0074	17.0	663	2.7532
3.0074	18.0	702	2.7404
3.0074	19.0	741	2.7270
3.0074	20.0	780	2.7158
3.0074	21.0	819	2.7039
3.0074	22.0	858	2.6945
3.0074	23.0	897	2.6824
3.0074	24.0	936	2.6743
3.0074	25.0	975	2.6640
2.7328	26.0	1014	2.6566
2.7328	27.0	1053	2.6495
2.7328	28.0	1092	2.6390
2.7328	29.0	1131	2.6308
2.7328	30.0	1170	2.6250
2.7328	31.0	1209	2.6164
2.7328	32.0	1248	2.6106
2.7328	33.0	1287	2.6037
2.7328	34.0	1326	2.5978
2.7328	35.0	1365	2.5913
2.7328	36.0	1404	2.5867
2.7328	37.0	1443	2.5820
2.7328	38.0	1482	2.5763
2.6263	39.0	1521	2.5725
2.6263	40.0	1560	2.5689
2.6263	41.0	1599	2.5636
2.6263	42.0	1638	2.5610
2.6263	43.0	1677	2.5590
2.6263	44.0	1716	2.5566
2.6263	45.0	1755	2.5549
2.6263	46.0	1794	2.5523
2.6263	47.0	1833	2.5512
2.6263	48.0	1872	2.5500
2.6263	49.0	1911	2.5496
2.6263	50.0	1950	2.5493

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.17.0
• Tokenizers 0.15.1