fine-tuned-DatasetQAS-Squad-ID-with-indobert-base-uncased-with-ITTL-with-freeze-LR-1e-05

This model is a fine-tuned version of indolem/indobert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.5175
• Exact Match: 48.5572
• F1: 65.0249

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 128
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1
2.0255	0.5	463	1.8578	38.8323	53.2780
1.8396	1.0	926	1.6659	43.2069	59.4121
1.6258	1.5	1389	1.5971	45.0913	61.6718
1.5939	2.0	1852	1.5523	46.3447	62.8415
1.4904	2.5	2315	1.5345	46.9589	63.7167
1.5015	3.0	2778	1.5060	47.4889	64.4261
1.3787	3.5	3241	1.5092	47.7833	64.2215
1.3629	4.0	3704	1.4885	48.0273	64.6938
1.3229	4.5	4167	1.5174	48.2712	64.9266
1.2848	5.0	4630	1.4942	48.4899	64.9576
1.2703	5.5	5093	1.5074	48.5657	65.0539
1.2104	6.0	5556	1.5112	48.1114	64.6513
1.1775	6.5	6019	1.5004	48.1534	64.8169
1.2303	7.0	6482	1.4956	48.4647	65.0723
1.1673	7.5	6945	1.5151	48.5825	65.0862
1.1771	8.0	7408	1.5057	48.5657	65.0123
1.1172	8.5	7871	1.5286	48.4311	64.7537
1.1282	9.0	8334	1.5175	48.5572	65.0249

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu117
• Datasets 2.2.0
• Tokenizers 0.13.2