bert-base-uncased-sst-2-16-13-smoothed

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6686
• Accuracy: 0.6875

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 50
• num_epochs: 75
• label_smoothing_factor: 0.45

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	1	0.7007	0.5
No log	2.0	2	0.7007	0.5
No log	3.0	3	0.7005	0.5
No log	4.0	4	0.7004	0.5
No log	5.0	5	0.7001	0.5
No log	6.0	6	0.6998	0.5
No log	7.0	7	0.6994	0.5
No log	8.0	8	0.6990	0.5
No log	9.0	9	0.6986	0.5
0.6996	10.0	10	0.6982	0.5
0.6996	11.0	11	0.6977	0.5
0.6996	12.0	12	0.6972	0.5
0.6996	13.0	13	0.6967	0.5312
0.6996	14.0	14	0.6962	0.5312
0.6996	15.0	15	0.6956	0.5
0.6996	16.0	16	0.6950	0.5312
0.6996	17.0	17	0.6944	0.5625
0.6996	18.0	18	0.6939	0.625
0.6996	19.0	19	0.6935	0.5938
0.6877	20.0	20	0.6930	0.5625
0.6877	21.0	21	0.6926	0.5625
0.6877	22.0	22	0.6920	0.5938
0.6877	23.0	23	0.6915	0.5938
0.6877	24.0	24	0.6909	0.5938
0.6877	25.0	25	0.6903	0.5938
0.6877	26.0	26	0.6897	0.5625
0.6877	27.0	27	0.6892	0.5625
0.6877	28.0	28	0.6888	0.5625
0.6877	29.0	29	0.6880	0.5625
0.6693	30.0	30	0.6873	0.5625
0.6693	31.0	31	0.6872	0.5625
0.6693	32.0	32	0.6869	0.5938
0.6693	33.0	33	0.6863	0.5625
0.6693	34.0	34	0.6857	0.5625
0.6693	35.0	35	0.6851	0.5625
0.6693	36.0	36	0.6844	0.5938
0.6693	37.0	37	0.6836	0.5938
0.6693	38.0	38	0.6825	0.5938
0.6693	39.0	39	0.6818	0.5625
0.6173	40.0	40	0.6815	0.5625
0.6173	41.0	41	0.6816	0.5625
0.6173	42.0	42	0.6812	0.5938
0.6173	43.0	43	0.6804	0.625
0.6173	44.0	44	0.6794	0.6562
0.6173	45.0	45	0.6781	0.6562
0.6173	46.0	46	0.6770	0.6562
0.6173	47.0	47	0.6761	0.6562
0.6173	48.0	48	0.6757	0.6562
0.6173	49.0	49	0.6746	0.6562
0.5541	50.0	50	0.6736	0.6562
0.5541	51.0	51	0.6730	0.6562
0.5541	52.0	52	0.6725	0.6562
0.5541	53.0	53	0.6703	0.6562
0.5541	54.0	54	0.6682	0.6562
0.5541	55.0	55	0.6671	0.6562
0.5541	56.0	56	0.6660	0.6562
0.5541	57.0	57	0.6652	0.6562
0.5541	58.0	58	0.6651	0.6562
0.5541	59.0	59	0.6652	0.6562
0.5403	60.0	60	0.6654	0.6875
0.5403	61.0	61	0.6660	0.6562
0.5403	62.0	62	0.6664	0.6562
0.5403	63.0	63	0.6666	0.6562
0.5403	64.0	64	0.6675	0.6562
0.5403	65.0	65	0.6684	0.6562
0.5403	66.0	66	0.6692	0.6562
0.5403	67.0	67	0.6699	0.6562
0.5403	68.0	68	0.6703	0.6562
0.5403	69.0	69	0.6705	0.6562
0.538	70.0	70	0.6706	0.6562
0.538	71.0	71	0.6703	0.6562
0.538	72.0	72	0.6696	0.6562
0.538	73.0	73	0.6691	0.6562
0.538	74.0	74	0.6688	0.6562
0.538	75.0	75	0.6686	0.6875

Framework versions

• Transformers 4.32.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.4.0
• Tokenizers 0.13.3