simonycl/roberta-base-sst-2-16-42-smoothed

bert-base-uncased-sst-2-16-42-smoothed

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6339
• Accuracy: 0.7812

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 50
• num_epochs: 75
• label_smoothing_factor: 0.45

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	4	0.7048	0.5
No log	2.0	8	0.7021	0.5
0.7078	3.0	12	0.6982	0.5
0.7078	4.0	16	0.6936	0.4922
0.6935	5.0	20	0.6898	0.5625
0.6935	6.0	24	0.6878	0.625
0.6935	7.0	28	0.6856	0.6484
0.682	8.0	32	0.6818	0.6797
0.682	9.0	36	0.6759	0.7109
0.6567	10.0	40	0.6692	0.6562
0.6567	11.0	44	0.6662	0.6562
0.6567	12.0	48	0.6669	0.6484
0.6177	13.0	52	0.6612	0.6719
0.6177	14.0	56	0.6589	0.6641
0.5787	15.0	60	0.6565	0.6797
0.5787	16.0	64	0.6519	0.7109
0.5787	17.0	68	0.6506	0.7266
0.5487	18.0	72	0.6496	0.75
0.5487	19.0	76	0.6489	0.7344
0.5398	20.0	80	0.6492	0.75
0.5398	21.0	84	0.6467	0.7578
0.5398	22.0	88	0.6439	0.75
0.5384	23.0	92	0.6419	0.75
0.5384	24.0	96	0.6442	0.75
0.5389	25.0	100	0.6442	0.7422
0.5389	26.0	104	0.6391	0.75
0.5389	27.0	108	0.6428	0.7734
0.5373	28.0	112	0.6449	0.7656
0.5373	29.0	116	0.6412	0.7812
0.5379	30.0	120	0.6359	0.7578
0.5379	31.0	124	0.6348	0.7734
0.5379	32.0	128	0.6343	0.7891
0.5366	33.0	132	0.6340	0.7969
0.5366	34.0	136	0.6340	0.7891
0.5366	35.0	140	0.6339	0.7734
0.5366	36.0	144	0.6337	0.7734
0.5366	37.0	148	0.6335	0.7734
0.5363	38.0	152	0.6334	0.7969
0.5363	39.0	156	0.6347	0.7734
0.5367	40.0	160	0.6355	0.7656
0.5367	41.0	164	0.6363	0.7578
0.5367	42.0	168	0.6374	0.7656
0.5359	43.0	172	0.6375	0.7656
0.5359	44.0	176	0.6357	0.7578
0.5358	45.0	180	0.6351	0.7656
0.5358	46.0	184	0.6339	0.7734
0.5358	47.0	188	0.6334	0.7812
0.5362	48.0	192	0.6330	0.7891
0.5362	49.0	196	0.6327	0.7891
0.5364	50.0	200	0.6329	0.7891
0.5364	51.0	204	0.6341	0.7734
0.5364	52.0	208	0.6337	0.7734
0.5365	53.0	212	0.6326	0.7891
0.5365	54.0	216	0.6325	0.7891
0.5361	55.0	220	0.6326	0.7969
0.5361	56.0	224	0.6328	0.7891
0.5361	57.0	228	0.6328	0.7891
0.5361	58.0	232	0.6330	0.7891
0.5361	59.0	236	0.6335	0.7812
0.5363	60.0	240	0.6340	0.7812
0.5363	61.0	244	0.6343	0.7812
0.5363	62.0	248	0.6346	0.7734
0.536	63.0	252	0.6348	0.7734
0.536	64.0	256	0.6349	0.7734
0.5362	65.0	260	0.6353	0.7656
0.5362	66.0	264	0.6358	0.7656
0.5362	67.0	268	0.6361	0.7578
0.536	68.0	272	0.6355	0.7656
0.536	69.0	276	0.6349	0.7734
0.5358	70.0	280	0.6344	0.7734
0.5358	71.0	284	0.6342	0.7812
0.5358	72.0	288	0.6340	0.7812
0.5357	73.0	292	0.6340	0.7812
0.5357	74.0	296	0.6339	0.7812
0.5358	75.0	300	0.6339	0.7812

Framework versions

• Transformers 4.32.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.4.0
• Tokenizers 0.13.3