roberta-base-sst-2-32-13-smoothed

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6023
• Accuracy: 0.8906

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 50
• num_epochs: 75
• label_smoothing_factor: 0.45

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	2	0.6943	0.5
No log	2.0	4	0.6942	0.5
No log	3.0	6	0.6941	0.5
No log	4.0	8	0.6939	0.5
0.695	5.0	10	0.6937	0.5
0.695	6.0	12	0.6935	0.5
0.695	7.0	14	0.6933	0.5
0.695	8.0	16	0.6932	0.5
0.695	9.0	18	0.6930	0.5
0.6959	10.0	20	0.6928	0.5
0.6959	11.0	22	0.6927	0.5156
0.6959	12.0	24	0.6926	0.6094
0.6959	13.0	26	0.6925	0.5781
0.6959	14.0	28	0.6923	0.5625
0.6919	15.0	30	0.6922	0.5625
0.6919	16.0	32	0.6920	0.5625
0.6919	17.0	34	0.6917	0.6094
0.6919	18.0	36	0.6913	0.5938
0.6919	19.0	38	0.6908	0.6406
0.6896	20.0	40	0.6902	0.7188
0.6896	21.0	42	0.6892	0.7812
0.6896	22.0	44	0.6878	0.6719
0.6896	23.0	46	0.6855	0.7344
0.6896	24.0	48	0.6816	0.7344
0.6745	25.0	50	0.6737	0.7812
0.6745	26.0	52	0.6571	0.8438
0.6745	27.0	54	0.6290	0.8438
0.6745	28.0	56	0.6161	0.8438
0.6745	29.0	58	0.6202	0.8594
0.5833	30.0	60	0.6190	0.875
0.5833	31.0	62	0.6210	0.8594
0.5833	32.0	64	0.6147	0.8594
0.5833	33.0	66	0.6056	0.9062
0.5833	34.0	68	0.6082	0.9062
0.5433	35.0	70	0.6194	0.875
0.5433	36.0	72	0.6035	0.9062
0.5433	37.0	74	0.5986	0.8906
0.5433	38.0	76	0.5970	0.8906
0.5433	39.0	78	0.6038	0.8906
0.5402	40.0	80	0.6061	0.8906
0.5402	41.0	82	0.6018	0.8906
0.5402	42.0	84	0.6013	0.9062
0.5402	43.0	86	0.6018	0.8906
0.5402	44.0	88	0.6086	0.8594
0.5384	45.0	90	0.6100	0.8594
0.5384	46.0	92	0.6044	0.8906
0.5384	47.0	94	0.6022	0.8906
0.5384	48.0	96	0.6007	0.8906
0.5384	49.0	98	0.6003	0.8906
0.5368	50.0	100	0.6013	0.8906
0.5368	51.0	102	0.6012	0.8906
0.5368	52.0	104	0.6006	0.8906
0.5368	53.0	106	0.6005	0.8906
0.5368	54.0	108	0.6011	0.8906
0.537	55.0	110	0.6013	0.8906
0.537	56.0	112	0.6014	0.8906
0.537	57.0	114	0.6013	0.9062
0.537	58.0	116	0.6011	0.9062
0.537	59.0	118	0.6006	0.9062
0.5364	60.0	120	0.5999	0.9062
0.5364	61.0	122	0.5994	0.9062
0.5364	62.0	124	0.5991	0.9062
0.5364	63.0	126	0.5992	0.9062
0.5364	64.0	128	0.5996	0.9062
0.5362	65.0	130	0.6000	0.9062
0.5362	66.0	132	0.6004	0.9062
0.5362	67.0	134	0.6007	0.9062
0.5362	68.0	136	0.6015	0.9062
0.5362	69.0	138	0.6020	0.9062
0.5362	70.0	140	0.6020	0.9062
0.5362	71.0	142	0.6021	0.9062
0.5362	72.0	144	0.6023	0.8906
0.5362	73.0	146	0.6023	0.8906
0.5362	74.0	148	0.6023	0.8906
0.536	75.0	150	0.6023	0.8906

Framework versions

• Transformers 4.32.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.4.0
• Tokenizers 0.13.3