Ranjit/massive_syn

massive_indo

This model is a fine-tuned version of xxxxxxxxx on the massive dataset. It achieves the following results on the evaluation set:

• Loss: 0.4772
• F1: 0.9465

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	F1
4.5227	0.7	100	4.4260	0.0998
3.9734	1.41	200	3.9864	0.2616
3.6121	2.11	300	3.5977	0.3561
3.2481	2.82	400	3.2366	0.4929
2.9212	3.52	500	2.9196	0.5677
2.5236	4.23	600	2.6016	0.6625
2.3089	4.93	700	2.3252	0.7196
2.0311	5.63	800	2.0840	0.7740
1.793	6.34	900	1.8612	0.8168
1.5913	7.04	1000	1.6588	0.8524
1.4071	7.75	1100	1.4833	0.8781
1.2169	8.45	1200	1.3214	0.8806
1.1085	9.15	1300	1.2012	0.8983
0.9752	9.86	1400	1.0659	0.9145
0.8891	10.56	1500	0.9672	0.9165
0.7205	11.27	1600	0.8726	0.9225
0.6869	11.97	1700	0.8065	0.9320
0.5818	12.68	1800	0.7458	0.9367
0.5437	13.38	1900	0.6843	0.9374
0.4759	14.08	2000	0.6411	0.9357
0.4674	14.79	2100	0.6010	0.9388
0.4305	15.49	2200	0.5682	0.9427
0.3862	16.2	2300	0.5369	0.9424
0.3564	16.9	2400	0.5208	0.9412
0.3292	17.61	2500	0.5022	0.9424
0.3237	18.31	2600	0.4903	0.9450
0.3187	19.01	2700	0.4807	0.9460
0.317	19.72	2800	0.4772	0.9465

Framework versions

• Transformers 4.33.3
• Pytorch 2.0.1+cu118
• Tokenizers 0.13.3