roberta-base-sst-2-16-13-30

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6585
• Accuracy: 0.6875

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1.5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 5
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	1	0.6934	0.5
No log	2.0	2	0.6933	0.5
No log	3.0	3	0.6933	0.5
No log	4.0	4	0.6929	0.5
No log	5.0	5	0.6925	0.5
No log	6.0	6	0.6920	0.5
No log	7.0	7	0.6914	0.5
No log	8.0	8	0.6909	0.6875
No log	9.0	9	0.6904	0.625
0.6897	10.0	10	0.6899	0.5
0.6897	11.0	11	0.6894	0.5
0.6897	12.0	12	0.6888	0.5
0.6897	13.0	13	0.6880	0.5312
0.6897	14.0	14	0.6871	0.5312
0.6897	15.0	15	0.6860	0.5312
0.6897	16.0	16	0.6849	0.6562
0.6897	17.0	17	0.6836	0.7188
0.6897	18.0	18	0.6821	0.6875
0.6897	19.0	19	0.6805	0.6875
0.6642	20.0	20	0.6788	0.6875
0.6642	21.0	21	0.6768	0.7188
0.6642	22.0	22	0.6746	0.7188
0.6642	23.0	23	0.6723	0.7188
0.6642	24.0	24	0.6696	0.7188
0.6642	25.0	25	0.6670	0.6875
0.6642	26.0	26	0.6644	0.6875
0.6642	27.0	27	0.6622	0.7188
0.6642	28.0	28	0.6604	0.7188
0.6642	29.0	29	0.6592	0.6875
0.5945	30.0	30	0.6585	0.6875

Framework versions

• Transformers 4.32.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.4.0
• Tokenizers 0.13.3