my_awesome_eli5_mlm_model

This model is a fine-tuned version of bert-base-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.5349

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	38	3.3747
No log	2.0	76	3.1852
No log	3.0	114	3.0839
No log	4.0	152	3.1410
No log	5.0	190	3.0394
No log	6.0	228	3.0631
No log	7.0	266	3.1484
No log	8.0	304	2.7834
No log	9.0	342	2.9527
No log	10.0	380	3.2091
No log	11.0	418	3.0497
No log	12.0	456	2.7234
No log	13.0	494	2.7865
2.8163	14.0	532	2.5425
2.8163	15.0	570	2.9351
2.8163	16.0	608	2.9367
2.8163	17.0	646	3.0570
2.8163	18.0	684	2.8902
2.8163	19.0	722	2.8285
2.8163	20.0	760	3.1220
2.8163	21.0	798	2.5891
2.8163	22.0	836	2.7217
2.8163	23.0	874	2.8660
2.8163	24.0	912	2.9048
2.8163	25.0	950	2.5349

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.1