distilbert-base-uncased-finetuned-diabetes_sentences

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5278
• Accuracy: 0.8462
• F1: 0.8441

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 200

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
1.1171	1.0	2	1.1097	0.4103	0.3239
1.0594	2.0	4	1.0910	0.5641	0.4791
1.0633	3.0	6	1.0726	0.5897	0.4859
1.0348	4.0	8	1.0520	0.6410	0.5779
0.9992	5.0	10	1.0326	0.5385	0.4980
0.9915	6.0	12	1.0260	0.4872	0.4518
0.9447	7.0	14	0.9811	0.5641	0.5369
0.8217	8.0	16	0.9087	0.8205	0.8205
0.8067	9.0	18	0.8497	0.8462	0.8437
0.7156	10.0	20	0.8001	0.8462	0.8437
0.6859	11.0	22	0.7691	0.8462	0.8437
0.5988	12.0	24	0.7399	0.8462	0.8437
0.5365	13.0	26	0.6851	0.8462	0.8437
0.4467	14.0	28	0.6255	0.8462	0.8437
0.4347	15.0	30	0.5791	0.8462	0.8437
0.363	16.0	32	0.5482	0.8462	0.8437
0.2946	17.0	34	0.5359	0.7949	0.7967
0.2343	18.0	36	0.4981	0.7949	0.7967
0.1999	19.0	38	0.4467	0.8718	0.8706
0.1615	20.0	40	0.4282	0.8718	0.8706
0.1314	21.0	42	0.4236	0.8718	0.8706
0.1386	22.0	44	0.4183	0.8718	0.8706
0.0973	23.0	46	0.4291	0.8462	0.8467
0.0853	24.0	48	0.4173	0.8462	0.8467
0.0732	25.0	50	0.3749	0.8462	0.8467
0.0641	26.0	52	0.3341	0.8974	0.8971
0.0541	27.0	54	0.3223	0.8974	0.8971
0.0481	28.0	56	0.3277	0.8974	0.8971
0.0383	29.0	58	0.3415	0.8974	0.8971
0.036	30.0	60	0.3609	0.8974	0.8971
0.0299	31.0	62	0.3823	0.8974	0.8971
0.0321	32.0	64	0.4026	0.8974	0.8971
0.03	33.0	66	0.4176	0.8718	0.8706
0.0277	34.0	68	0.4201	0.8718	0.8706
0.0236	35.0	70	0.4129	0.8718	0.8706
0.022	36.0	72	0.4003	0.8974	0.8971
0.022	37.0	74	0.3865	0.8974	0.8971
0.0211	38.0	76	0.3731	0.8974	0.8971
0.017	39.0	78	0.3634	0.8718	0.8705
0.0188	40.0	80	0.3618	0.8718	0.8705
0.0169	41.0	82	0.3683	0.8718	0.8705
0.0161	42.0	84	0.3810	0.8718	0.8705
0.0162	43.0	86	0.3944	0.8718	0.8705
0.0141	44.0	88	0.4091	0.8974	0.8971
0.0132	45.0	90	0.4233	0.8974	0.8971
0.0143	46.0	92	0.4335	0.8718	0.8706
0.0142	47.0	94	0.4413	0.8718	0.8706
0.0125	48.0	96	0.4436	0.8718	0.8706
0.0115	49.0	98	0.4437	0.8718	0.8706
0.0106	50.0	100	0.4410	0.8462	0.8441
0.0109	51.0	102	0.4376	0.8462	0.8441
0.0119	52.0	104	0.4341	0.8462	0.8441
0.012	53.0	106	0.4322	0.8718	0.8705
0.0122	54.0	108	0.4314	0.8718	0.8705
0.0107	55.0	110	0.4315	0.8718	0.8705
0.0102	56.0	112	0.4324	0.8718	0.8705
0.0102	57.0	114	0.4351	0.8462	0.8441
0.0098	58.0	116	0.4379	0.8462	0.8441
0.009	59.0	118	0.4399	0.8462	0.8441
0.0099	60.0	120	0.4415	0.8462	0.8441
0.0094	61.0	122	0.4429	0.8462	0.8441
0.008	62.0	124	0.4479	0.8462	0.8441
0.0084	63.0	126	0.4531	0.8462	0.8441
0.0079	64.0	128	0.4571	0.8462	0.8441
0.0079	65.0	130	0.4607	0.8462	0.8441
0.0076	66.0	132	0.4637	0.8462	0.8441
0.0072	67.0	134	0.4659	0.8462	0.8441
0.0076	68.0	136	0.4693	0.8462	0.8441
0.0078	69.0	138	0.4726	0.8462	0.8441
0.0066	70.0	140	0.4729	0.8462	0.8441
0.0082	71.0	142	0.4711	0.8462	0.8441
0.0075	72.0	144	0.4673	0.8462	0.8441
0.0065	73.0	146	0.4645	0.8462	0.8441
0.0064	74.0	148	0.4623	0.8462	0.8441
0.0075	75.0	150	0.4613	0.8718	0.8705
0.0064	76.0	152	0.4616	0.8718	0.8705
0.0063	77.0	154	0.4627	0.8462	0.8441
0.0072	78.0	156	0.4635	0.8462	0.8441
0.0058	79.0	158	0.4636	0.8462	0.8441
0.006	80.0	160	0.4641	0.8462	0.8441
0.0061	81.0	162	0.4651	0.8462	0.8441
0.0054	82.0	164	0.4675	0.8462	0.8441
0.0066	83.0	166	0.4692	0.8462	0.8441
0.0056	84.0	168	0.4699	0.8462	0.8441
0.0058	85.0	170	0.4706	0.8462	0.8441
0.0056	86.0	172	0.4718	0.8462	0.8441
0.005	87.0	174	0.4745	0.8462	0.8441
0.0062	88.0	176	0.4766	0.8462	0.8441
0.0052	89.0	178	0.4786	0.8462	0.8441
0.0055	90.0	180	0.4801	0.8462	0.8441
0.0052	91.0	182	0.4811	0.8462	0.8441
0.0052	92.0	184	0.4818	0.8462	0.8441
0.0057	93.0	186	0.4832	0.8462	0.8441
0.005	94.0	188	0.4844	0.8462	0.8441
0.0055	95.0	190	0.4850	0.8462	0.8441
0.005	96.0	192	0.4852	0.8462	0.8441
0.0055	97.0	194	0.4860	0.8462	0.8441
0.0047	98.0	196	0.4872	0.8462	0.8441
0.0043	99.0	198	0.4889	0.8462	0.8441
0.0049	100.0	200	0.4902	0.8462	0.8441
0.0048	101.0	202	0.4909	0.8462	0.8441
0.0044	102.0	204	0.4908	0.8462	0.8441
0.004	103.0	206	0.4915	0.8462	0.8441
0.0044	104.0	208	0.4918	0.8462	0.8441
0.0044	105.0	210	0.4935	0.8462	0.8441
0.0043	106.0	212	0.4956	0.8462	0.8441
0.004	107.0	214	0.4978	0.8462	0.8441
0.0047	108.0	216	0.4987	0.8462	0.8441
0.0037	109.0	218	0.4994	0.8462	0.8441
0.0046	110.0	220	0.5012	0.8462	0.8441
0.004	111.0	222	0.5021	0.8462	0.8441
0.004	112.0	224	0.5030	0.8462	0.8441
0.004	113.0	226	0.5044	0.8462	0.8441
0.0039	114.0	228	0.5053	0.8462	0.8441
0.0038	115.0	230	0.5058	0.8462	0.8441
0.0041	116.0	232	0.5054	0.8462	0.8441
0.0038	117.0	234	0.5047	0.8462	0.8441
0.0035	118.0	236	0.5043	0.8462	0.8441
0.004	119.0	238	0.5035	0.8462	0.8441
0.0039	120.0	240	0.5029	0.8462	0.8441
0.0036	121.0	242	0.5019	0.8462	0.8441
0.0042	122.0	244	0.5012	0.8462	0.8441
0.0033	123.0	246	0.5005	0.8462	0.8441
0.0034	124.0	248	0.5003	0.8462	0.8441
0.0038	125.0	250	0.5002	0.8462	0.8441
0.0035	126.0	252	0.4998	0.8462	0.8441
0.0033	127.0	254	0.5002	0.8462	0.8441
0.0041	128.0	256	0.5010	0.8462	0.8441
0.0036	129.0	258	0.5025	0.8462	0.8441
0.0036	130.0	260	0.5037	0.8462	0.8441
0.0032	131.0	262	0.5049	0.8462	0.8441
0.0033	132.0	264	0.5061	0.8462	0.8441
0.0038	133.0	266	0.5075	0.8462	0.8441
0.0041	134.0	268	0.5087	0.8462	0.8441
0.0034	135.0	270	0.5094	0.8462	0.8441
0.0032	136.0	272	0.5107	0.8462	0.8441
0.0035	137.0	274	0.5123	0.8462	0.8441
0.0032	138.0	276	0.5138	0.8462	0.8441
0.0031	139.0	278	0.5143	0.8462	0.8441
0.0034	140.0	280	0.5145	0.8462	0.8441
0.0036	141.0	282	0.5151	0.8462	0.8441
0.003	142.0	284	0.5160	0.8462	0.8441
0.0034	143.0	286	0.5162	0.8462	0.8441
0.0031	144.0	288	0.5160	0.8462	0.8441
0.0031	145.0	290	0.5157	0.8462	0.8441
0.0032	146.0	292	0.5155	0.8462	0.8441
0.0029	147.0	294	0.5159	0.8462	0.8441
0.0032	148.0	296	0.5162	0.8462	0.8441
0.0036	149.0	298	0.5164	0.8462	0.8441
0.0028	150.0	300	0.5167	0.8462	0.8441
0.0026	151.0	302	0.5172	0.8462	0.8441
0.0028	152.0	304	0.5174	0.8462	0.8441
0.0031	153.0	306	0.5172	0.8462	0.8441
0.0029	154.0	308	0.5168	0.8462	0.8441
0.0031	155.0	310	0.5168	0.8462	0.8441
0.0033	156.0	312	0.5167	0.8462	0.8441
0.003	157.0	314	0.5168	0.8462	0.8441
0.0029	158.0	316	0.5175	0.8462	0.8441
0.0031	159.0	318	0.5181	0.8462	0.8441
0.003	160.0	320	0.5186	0.8462	0.8441
0.0031	161.0	322	0.5190	0.8462	0.8441
0.0032	162.0	324	0.5194	0.8462	0.8441
0.0028	163.0	326	0.5201	0.8462	0.8441
0.0026	164.0	328	0.5209	0.8462	0.8441
0.0032	165.0	330	0.5218	0.8462	0.8441
0.0031	166.0	332	0.5226	0.8462	0.8441
0.0029	167.0	334	0.5234	0.8462	0.8441
0.0032	168.0	336	0.5239	0.8462	0.8441
0.0031	169.0	338	0.5240	0.8462	0.8441
0.003	170.0	340	0.5243	0.8462	0.8441
0.0031	171.0	342	0.5246	0.8462	0.8441
0.0024	172.0	344	0.5250	0.8462	0.8441
0.0025	173.0	346	0.5256	0.8462	0.8441
0.0028	174.0	348	0.5265	0.8462	0.8441
0.003	175.0	350	0.5272	0.8462	0.8441
0.003	176.0	352	0.5275	0.8462	0.8441
0.0027	177.0	354	0.5278	0.8462	0.8441
0.0027	178.0	356	0.5277	0.8462	0.8441
0.0028	179.0	358	0.5276	0.8462	0.8441
0.0027	180.0	360	0.5274	0.8462	0.8441
0.0028	181.0	362	0.5272	0.8462	0.8441
0.0035	182.0	364	0.5270	0.8462	0.8441
0.003	183.0	366	0.5269	0.8462	0.8441
0.0028	184.0	368	0.5267	0.8462	0.8441
0.0026	185.0	370	0.5266	0.8462	0.8441
0.0033	186.0	372	0.5265	0.8462	0.8441
0.0028	187.0	374	0.5265	0.8462	0.8441
0.0025	188.0	376	0.5267	0.8462	0.8441
0.0029	189.0	378	0.5268	0.8462	0.8441
0.0029	190.0	380	0.5269	0.8462	0.8441
0.0024	191.0	382	0.5270	0.8462	0.8441
0.0031	192.0	384	0.5271	0.8462	0.8441
0.0028	193.0	386	0.5273	0.8462	0.8441
0.0026	194.0	388	0.5274	0.8462	0.8441
0.0027	195.0	390	0.5275	0.8462	0.8441
0.0026	196.0	392	0.5276	0.8462	0.8441
0.0026	197.0	394	0.5277	0.8462	0.8441
0.0028	198.0	396	0.5277	0.8462	0.8441
0.0026	199.0	398	0.5278	0.8462	0.8441
0.003	200.0	400	0.5278	0.8462	0.8441

Framework versions

• Transformers 4.28.1
• Pytorch 2.0.0+cu118
• Datasets 2.12.0
• Tokenizers 0.13.3