distilbert-ner-qlorafinetune-runs

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1707
• Precision: 0.9584
• Recall: 0.9495
• F1: 0.9539
• Accuracy: 0.9737

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0004
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 640
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
4.6043	0.0151	20	1.9157	0.0	0.0	0.0	0.6192
1.5758	0.0303	40	0.7695	0.7538	0.4952	0.5977	0.8411
0.391	0.0454	60	0.3487	0.8781	0.8772	0.8777	0.9548
0.2109	0.0605	80	0.2782	0.8970	0.9301	0.9133	0.9655
0.1793	0.0756	100	0.2435	0.9635	0.9318	0.9474	0.9664
0.1055	0.0908	120	0.2311	0.9614	0.9330	0.9470	0.9667
0.3157	0.1059	140	0.2210	0.9631	0.9333	0.9480	0.9677
0.1085	0.1210	160	0.2088	0.9336	0.9364	0.9350	0.9692
0.2085	0.1362	180	0.2044	0.9576	0.9351	0.9462	0.9695
0.3833	0.1513	200	0.1992	0.9478	0.9402	0.9440	0.9703
0.097	0.1664	220	0.1957	0.9482	0.9426	0.9454	0.9712
0.12	0.1815	240	0.1964	0.9541	0.9421	0.9481	0.9716
0.1696	0.1967	260	0.2697	0.9315	0.9444	0.9379	0.9718
0.1405	0.2118	280	0.1933	0.9691	0.9424	0.9555	0.9717
0.1992	0.2269	300	0.1887	0.9538	0.9444	0.9491	0.9722
0.0907	0.2421	320	0.1870	0.9629	0.9441	0.9534	0.9724
0.1778	0.2572	340	0.1852	0.9461	0.9471	0.9466	0.9730
0.1474	0.2723	360	0.1821	0.9467	0.9472	0.9469	0.9731
0.1972	0.2874	380	0.1798	0.9522	0.9472	0.9497	0.9731
0.1807	0.3026	400	0.1823	0.9646	0.9465	0.9555	0.9729
0.1388	0.3177	420	0.1771	0.9474	0.9491	0.9482	0.9733
0.1664	0.3328	440	0.1762	0.9655	0.9470	0.9562	0.9732
0.1559	0.3480	460	0.1747	0.9618	0.9482	0.9550	0.9733
0.233	0.3631	480	0.1750	0.9663	0.9484	0.9573	0.9733
0.1851	0.3782	500	0.1738	0.9616	0.9492	0.9554	0.9735
0.2512	0.3933	520	0.1725	0.9642	0.9491	0.9566	0.9736
0.0823	0.4085	540	0.1721	0.9624	0.9490	0.9556	0.9736
0.0865	0.4236	560	0.1717	0.9624	0.9491	0.9557	0.9737
0.0611	0.4387	580	0.1714	0.9607	0.9493	0.9550	0.9738
0.105	0.4539	600	0.1710	0.9612	0.9493	0.9552	0.9737
0.1192	0.4690	620	0.1709	0.9584	0.9495	0.9540	0.9737
0.1923	0.4841	640	0.1707	0.9584	0.9495	0.9539	0.9737

Framework versions

• PEFT 0.12.0
• Transformers 4.43.3
• Pytorch 2.4.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1