bert-base-uncased-sst-2-32-87

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.9995
• Accuracy: 0.875

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 150

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	2	1.3036	0.8281
No log	2.0	4	1.3032	0.8281
No log	3.0	6	1.3022	0.8281
No log	4.0	8	1.3002	0.8438
0.6888	5.0	10	1.2981	0.8438
0.6888	6.0	12	1.2958	0.8438
0.6888	7.0	14	1.2937	0.8438
0.6888	8.0	16	1.2916	0.8438
0.6888	9.0	18	1.2896	0.8281
0.6235	10.0	20	1.2880	0.8281
0.6235	11.0	22	1.2862	0.8281
0.6235	12.0	24	1.2847	0.8281
0.6235	13.0	26	1.2833	0.8281
0.6235	14.0	28	1.2827	0.8281
0.6224	15.0	30	1.2813	0.8281
0.6224	16.0	32	1.2788	0.8281
0.6224	17.0	34	1.2739	0.8281
0.6224	18.0	36	1.2670	0.8281
0.6224	19.0	38	1.2583	0.8281
0.5366	20.0	40	1.2501	0.8281
0.5366	21.0	42	1.2366	0.8281
0.5366	22.0	44	1.2258	0.8281
0.5366	23.0	46	1.2148	0.8281
0.5366	24.0	48	1.2069	0.8281
0.3634	25.0	50	1.1973	0.8281
0.3634	26.0	52	1.1888	0.8281
0.3634	27.0	54	1.1754	0.8281
0.3634	28.0	56	1.1583	0.8281
0.3634	29.0	58	1.1462	0.8281
0.3447	30.0	60	1.1399	0.8281
0.3447	31.0	62	1.1399	0.8281
0.3447	32.0	64	1.1328	0.8281
0.3447	33.0	66	1.1304	0.8281
0.3447	34.0	68	1.1275	0.8281
0.2231	35.0	70	1.1185	0.8281
0.2231	36.0	72	1.1059	0.8281
0.2231	37.0	74	1.0901	0.8281
0.2231	38.0	76	1.0711	0.8281
0.2231	39.0	78	1.0516	0.8281
0.0925	40.0	80	1.0339	0.8281
0.0925	41.0	82	1.0151	0.8281
0.0925	42.0	84	0.9910	0.8281
0.0925	43.0	86	0.9616	0.8281
0.0925	44.0	88	0.9422	0.8281
0.024	45.0	90	0.9346	0.8281
0.024	46.0	92	0.9374	0.8281
0.024	47.0	94	0.9413	0.8438
0.024	48.0	96	0.9460	0.8438
0.024	49.0	98	0.9470	0.8438
0.0161	50.0	100	0.9483	0.8438
0.0161	51.0	102	0.9505	0.8438
0.0161	52.0	104	0.9534	0.8438
0.0161	53.0	106	0.9565	0.8438
0.0161	54.0	108	0.9591	0.8438
0.0003	55.0	110	0.9613	0.8438
0.0003	56.0	112	0.9609	0.8438
0.0003	57.0	114	0.9606	0.8438
0.0003	58.0	116	0.9597	0.8438
0.0003	59.0	118	0.9582	0.8438
0.0003	60.0	120	0.9572	0.8438
0.0003	61.0	122	0.9557	0.8438
0.0003	62.0	124	0.9563	0.8438
0.0003	63.0	126	0.9514	0.8438
0.0003	64.0	128	0.9487	0.8438
0.0006	65.0	130	0.9472	0.8438
0.0006	66.0	132	0.9472	0.8438
0.0006	67.0	134	0.9486	0.8438
0.0006	68.0	136	0.9471	0.8438
0.0006	69.0	138	0.9569	0.8438
0.0008	70.0	140	0.9658	0.8438
0.0008	71.0	142	0.9732	0.8438
0.0008	72.0	144	0.9792	0.8438
0.0008	73.0	146	0.9836	0.8438
0.0008	74.0	148	0.9813	0.8438
0.0003	75.0	150	0.9750	0.8281
0.0003	76.0	152	0.9712	0.8281
0.0003	77.0	154	0.9636	0.8281
0.0003	78.0	156	0.9525	0.8281
0.0003	79.0	158	0.9410	0.8281
0.001	80.0	160	0.9323	0.8438
0.001	81.0	162	0.9256	0.8438
0.001	82.0	164	0.9293	0.8438
0.001	83.0	166	0.9429	0.8281
0.001	84.0	168	0.9565	0.8281
0.0002	85.0	170	0.9687	0.8281
0.0002	86.0	172	0.9796	0.8281
0.0002	87.0	174	0.9900	0.8281
0.0002	88.0	176	0.9985	0.8281
0.0002	89.0	178	1.0049	0.8281
0.0002	90.0	180	1.0099	0.8281
0.0002	91.0	182	1.0139	0.8281
0.0002	92.0	184	1.0170	0.8281
0.0002	93.0	186	1.0196	0.8281
0.0002	94.0	188	1.0218	0.8281
0.0002	95.0	190	1.0236	0.8281
0.0002	96.0	192	1.0250	0.8281
0.0002	97.0	194	1.0258	0.8281
0.0002	98.0	196	1.0262	0.8281
0.0002	99.0	198	1.0266	0.8281
0.0002	100.0	200	1.0274	0.8281
0.0002	101.0	202	1.0280	0.8281
0.0002	102.0	204	1.0286	0.8281
0.0002	103.0	206	1.0293	0.8281
0.0002	104.0	208	1.0298	0.8281
0.0001	105.0	210	1.0303	0.8281
0.0001	106.0	212	1.0309	0.8281
0.0001	107.0	214	1.0315	0.8281
0.0001	108.0	216	1.0318	0.8281
0.0001	109.0	218	1.0182	0.8281
0.0025	110.0	220	0.9797	0.8281
0.0025	111.0	222	0.9486	0.8438
0.0025	112.0	224	0.9379	0.8594
0.0025	113.0	226	0.9381	0.8594
0.0025	114.0	228	0.9421	0.8594
0.0002	115.0	230	0.9449	0.8594
0.0002	116.0	232	0.9477	0.8594
0.0002	117.0	234	0.9504	0.8594
0.0002	118.0	236	0.9531	0.8594
0.0002	119.0	238	0.9563	0.8594
0.0002	120.0	240	0.9597	0.8438
0.0002	121.0	242	0.9630	0.8438
0.0002	122.0	244	0.9902	0.8438
0.0002	123.0	246	0.9989	0.8438
0.0002	124.0	248	1.0010	0.8281
0.0007	125.0	250	1.0085	0.8438
0.0007	126.0	252	1.0163	0.8438
0.0007	127.0	254	1.0225	0.8438
0.0007	128.0	256	1.0279	0.8594
0.0007	129.0	258	1.0322	0.8594
0.0001	130.0	260	1.0336	0.8594
0.0001	131.0	262	1.0348	0.8594
0.0001	132.0	264	1.0358	0.8594
0.0001	133.0	266	1.0367	0.8594
0.0001	134.0	268	1.0300	0.8438
0.0005	135.0	270	1.0190	0.8438
0.0005	136.0	272	1.0185	0.8281
0.0005	137.0	274	1.0266	0.8438
0.0005	138.0	276	1.0311	0.8438
0.0005	139.0	278	1.0318	0.8438
0.0001	140.0	280	1.0306	0.8438
0.0001	141.0	282	1.0295	0.8281
0.0001	142.0	284	1.0286	0.8438
0.0001	143.0	286	1.0278	0.8438
0.0001	144.0	288	1.0272	0.8438
0.0001	145.0	290	1.0268	0.8438
0.0001	146.0	292	1.0266	0.8438
0.0001	147.0	294	1.0264	0.8438
0.0001	148.0	296	1.0265	0.8438
0.0001	149.0	298	0.9917	0.8594
0.0002	150.0	300	0.9995	0.875

Framework versions

• Transformers 4.32.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.4.0
• Tokenizers 0.13.3