indobert_artha

This model is a fine-tuned version of indolem/indobert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.6470
• Balanced accuracy: 0.4809

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Balanced accuracy
1.4443	1.0	92	1.2032	0.125
1.112	2.0	184	0.9012	0.3006
0.8258	3.0	276	0.8611	0.4565
0.6286	4.0	368	0.7711	0.4397
0.4205	5.0	460	0.8665	0.4900
0.3025	6.0	552	0.9085	0.4572
0.1904	7.0	644	1.1407	0.4584
0.1387	8.0	736	1.2191	0.4682
0.1294	9.0	828	1.3164	0.4470
0.097	10.0	920	1.4438	0.4245
0.0843	11.0	1012	1.3584	0.4603
0.0829	12.0	1104	1.3619	0.4442
0.0667	13.0	1196	1.4805	0.4536
0.0596	14.0	1288	1.6224	0.4917
0.0538	15.0	1380	1.6581	0.4253
0.0488	16.0	1472	1.6128	0.4982
0.0442	17.0	1564	1.8136	0.4951
0.0426	18.0	1656	1.6496	0.4859
0.0384	19.0	1748	1.6517	0.4702
0.0311	20.0	1840	1.6183	0.4901
0.0288	21.0	1932	1.7072	0.4647
0.0283	22.0	2024	1.6827	0.4653
0.0244	23.0	2116	1.6211	0.4777
0.0264	24.0	2208	1.6428	0.4719
0.0202	25.0	2300	1.6462	0.4907
0.0198	26.0	2392	1.6719	0.4841
0.024	27.0	2484	1.6376	0.4957
0.0205	28.0	2576	1.6477	0.4775
0.0162	29.0	2668	1.6459	0.4909
0.0165	30.0	2760	1.6470	0.4809

Framework versions

• Transformers 4.41.2
• Pytorch 2.1.2
• Datasets 2.20.0
• Tokenizers 0.19.1