roberta-large-ner-qlorafinetune-runs

This model is a fine-tuned version of FacebookAI/xlm-roberta-large on the conll2002 dataset. It achieves the following results on the evaluation set:

• Loss: 0.0853
• Precision: 0.8767
• Recall: 0.8819
• F1: 0.8793
• Accuracy: 0.9814

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0004
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Use paged_adamw_8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• training_steps: 1820
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
0.9933	0.0766	20	0.4461	0.1132	0.0565	0.0754	0.8691
0.2822	0.1533	40	0.2098	0.5212	0.6337	0.5720	0.9451
0.146	0.2299	60	0.1251	0.7357	0.7652	0.7502	0.9657
0.0986	0.3065	80	0.1222	0.7928	0.8134	0.8030	0.9720
0.1087	0.3831	100	0.1080	0.7738	0.7953	0.7844	0.9714
0.0874	0.4598	120	0.0974	0.8118	0.8205	0.8161	0.9742
0.0876	0.5364	140	0.0791	0.8374	0.8532	0.8452	0.9791
0.0909	0.6130	160	0.0831	0.8333	0.8486	0.8408	0.9779
0.0835	0.6897	180	0.0802	0.8452	0.8479	0.8465	0.9773
0.0706	0.7663	200	0.0879	0.8234	0.8359	0.8296	0.9757
0.0634	0.8429	220	0.0812	0.8368	0.8504	0.8435	0.9782
0.0548	0.9195	240	0.0918	0.8067	0.8235	0.8150	0.9741
0.0602	0.9962	260	0.0903	0.8328	0.8458	0.8393	0.9762
0.0528	1.0728	280	0.0878	0.8125	0.8456	0.8287	0.9760
0.0513	1.1494	300	0.0991	0.8269	0.8332	0.8300	0.9754
0.0565	1.2261	320	0.0805	0.8575	0.8683	0.8629	0.9788
0.0514	1.3027	340	0.0864	0.8104	0.8359	0.8230	0.9751
0.0499	1.3793	360	0.1011	0.8012	0.8194	0.8102	0.9740
0.0585	1.4559	380	0.0945	0.8179	0.8258	0.8219	0.9739
0.0489	1.5326	400	0.0887	0.8177	0.8389	0.8282	0.9755
0.0534	1.6092	420	0.0747	0.8283	0.8523	0.8401	0.9784
0.0612	1.6858	440	0.0743	0.8336	0.8532	0.8433	0.9775
0.0531	1.7625	460	0.0722	0.8632	0.8699	0.8666	0.9802
0.0453	1.8391	480	0.0752	0.8346	0.8502	0.8423	0.9776
0.0504	1.9157	500	0.0754	0.8373	0.8562	0.8466	0.9783
0.0468	1.9923	520	0.0734	0.8534	0.8612	0.8573	0.9794
0.0573	2.0690	540	0.0728	0.8412	0.8653	0.8531	0.9785
0.0344	2.1456	560	0.0741	0.8549	0.8690	0.8619	0.9794
0.0303	2.2222	580	0.0767	0.8428	0.8658	0.8541	0.9783
0.0421	2.2989	600	0.0753	0.8370	0.8587	0.8477	0.9786
0.0335	2.3755	620	0.0762	0.8523	0.8594	0.8558	0.9788
0.0376	2.4521	640	0.0735	0.8511	0.8732	0.8620	0.9797
0.0366	2.5287	660	0.0764	0.8647	0.8619	0.8633	0.9792
0.0351	2.6054	680	0.0771	0.8711	0.8713	0.8712	0.9795
0.0389	2.6820	700	0.0734	0.8563	0.8640	0.8601	0.9788
0.037	2.7586	720	0.0708	0.8475	0.8660	0.8567	0.9804
0.0434	2.8352	740	0.0719	0.8441	0.8587	0.8513	0.9783
0.0399	2.9119	760	0.0681	0.8572	0.8773	0.8671	0.9807
0.0381	2.9885	780	0.0679	0.8615	0.8759	0.8686	0.9809
0.0343	3.0651	800	0.0680	0.8593	0.8844	0.8717	0.9816
0.0283	3.1418	820	0.0708	0.8760	0.8780	0.8770	0.9812
0.0286	3.2184	840	0.0757	0.8688	0.8796	0.8742	0.9802
0.0279	3.2950	860	0.0721	0.875	0.8798	0.8774	0.9810
0.0274	3.3716	880	0.0717	0.8651	0.8766	0.8708	0.9796
0.0296	3.4483	900	0.0697	0.8558	0.8782	0.8669	0.9796
0.0355	3.5249	920	0.0624	0.8741	0.8856	0.8798	0.9820
0.0223	3.6015	940	0.0751	0.8693	0.8803	0.8748	0.9814
0.0265	3.6782	960	0.0785	0.8694	0.875	0.8722	0.9798
0.0261	3.7548	980	0.0701	0.8768	0.8830	0.8799	0.9808
0.0326	3.8314	1000	0.0693	0.8708	0.8842	0.8774	0.9820
0.028	3.9080	1020	0.0719	0.8579	0.8752	0.8665	0.9795
0.032	3.9847	1040	0.0726	0.8766	0.8801	0.8783	0.9809
0.0176	4.0613	1060	0.0766	0.8753	0.8821	0.8787	0.9809
0.0218	4.1379	1080	0.0808	0.8681	0.8805	0.8743	0.9810
0.0163	4.2146	1100	0.0833	0.8682	0.875	0.8716	0.9803
0.0201	4.2912	1120	0.0882	0.8702	0.8752	0.8727	0.9802
0.0193	4.3678	1140	0.0838	0.8678	0.8778	0.8727	0.9801
0.023	4.4444	1160	0.0855	0.8648	0.8761	0.8704	0.9802
0.0241	4.5211	1180	0.0793	0.8585	0.8768	0.8676	0.9795
0.0224	4.5977	1200	0.0806	0.8792	0.8863	0.8827	0.9814
0.0263	4.6743	1220	0.0737	0.8674	0.8775	0.8724	0.9816
0.0266	4.7510	1240	0.0824	0.8501	0.8706	0.8603	0.9794
0.0212	4.8276	1260	0.0770	0.8657	0.8709	0.8683	0.9811
0.0228	4.9042	1280	0.0724	0.8673	0.8771	0.8722	0.9810
0.0216	4.9808	1300	0.0703	0.8747	0.8842	0.8794	0.9817
0.0155	5.0575	1320	0.0782	0.8799	0.8904	0.8851	0.9824
0.015	5.1341	1340	0.0792	0.8822	0.8842	0.8832	0.9822
0.0159	5.2107	1360	0.0787	0.8771	0.8872	0.8821	0.9818
0.0182	5.2874	1380	0.0766	0.8767	0.8821	0.8794	0.9816
0.0147	5.3640	1400	0.0756	0.8753	0.8778	0.8765	0.9814
0.0182	5.4406	1420	0.0813	0.8755	0.8794	0.8775	0.9809
0.0163	5.5172	1440	0.0822	0.8823	0.8835	0.8829	0.9815
0.0172	5.5939	1460	0.0819	0.8767	0.8810	0.8789	0.9813
0.015	5.6705	1480	0.0777	0.8714	0.8796	0.8755	0.9811
0.0108	5.7471	1500	0.0801	0.8801	0.8835	0.8818	0.9814
0.0169	5.8238	1520	0.0798	0.8793	0.8853	0.8823	0.9818
0.016	5.9004	1540	0.0832	0.8723	0.8771	0.8747	0.9810
0.0128	5.9770	1560	0.0829	0.8730	0.8780	0.8755	0.9809
0.0101	6.0536	1580	0.0834	0.8711	0.8773	0.8742	0.9808
0.0124	6.1303	1600	0.0831	0.8710	0.8796	0.8753	0.9808
0.012	6.2069	1620	0.0855	0.8702	0.8761	0.8731	0.9805
0.0124	6.2835	1640	0.0822	0.8719	0.8805	0.8762	0.9811
0.0126	6.3602	1660	0.0823	0.8762	0.8830	0.8796	0.9812
0.0123	6.4368	1680	0.0814	0.8769	0.8853	0.8811	0.9812
0.0098	6.5134	1700	0.0827	0.8732	0.8828	0.8780	0.9813
0.0126	6.5900	1720	0.0820	0.8741	0.8810	0.8775	0.9810
0.0108	6.6667	1740	0.0840	0.8778	0.8830	0.8804	0.9815
0.0118	6.7433	1760	0.0852	0.8757	0.8824	0.8790	0.9814
0.0103	6.8199	1780	0.0854	0.8773	0.8824	0.8798	0.9814
0.0103	6.8966	1800	0.0854	0.8782	0.8828	0.8805	0.9815
0.0081	6.9732	1820	0.0853	0.8767	0.8819	0.8793	0.9814

Framework versions

• PEFT 0.13.2
• Transformers 4.46.3
• Pytorch 2.5.1
• Datasets 3.1.0
• Tokenizers 0.20.3