roberta-base-topic_classification_simple2

This model is a fine-tuned version of roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1250
• Accuracy: {'accuracy': 0.866996699669967}
• F1: {'f1': 0.8657113367537151}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	1.0	313	0.5920	{'accuracy': 0.8158415841584158}	{'f1': 0.8063426391052376}
0.7507	2.0	626	0.5183	{'accuracy': 0.8419141914191419}	{'f1': 0.8450438669495921}
0.7507	3.0	939	0.5089	{'accuracy': 0.8514851485148515}	{'f1': 0.8522994355907825}
0.3199	4.0	1252	0.6030	{'accuracy': 0.8508250825082508}	{'f1': 0.8484331857141633}
0.1504	5.0	1565	0.6894	{'accuracy': 0.8617161716171617}	{'f1': 0.8599694556754336}
0.1504	6.0	1878	0.8381	{'accuracy': 0.8448844884488449}	{'f1': 0.8461993387843019}
0.0822	7.0	2191	0.8515	{'accuracy': 0.8554455445544554}	{'f1': 0.8542784950089077}
0.0551	8.0	2504	0.9319	{'accuracy': 0.8531353135313532}	{'f1': 0.853451943641699}
0.0551	9.0	2817	0.9478	{'accuracy': 0.8577557755775578}	{'f1': 0.8565849659994866}
0.0377	10.0	3130	0.9998	{'accuracy': 0.8554455445544554}	{'f1': 0.8550659197552203}
0.0377	11.0	3443	1.0025	{'accuracy': 0.8554455445544554}	{'f1': 0.8550137537621838}
0.0279	12.0	3756	1.0728	{'accuracy': 0.8574257425742574}	{'f1': 0.8566278925949554}
0.0132	13.0	4069	1.0873	{'accuracy': 0.8623762376237624}	{'f1': 0.8610125122049608}
0.0132	14.0	4382	1.0989	{'accuracy': 0.8653465346534653}	{'f1': 0.863969705278768}
0.0124	15.0	4695	1.1379	{'accuracy': 0.8643564356435643}	{'f1': 0.8630599594036119}
0.0095	16.0	5008	1.1207	{'accuracy': 0.8653465346534653}	{'f1': 0.8639194427774014}
0.0095	17.0	5321	1.1053	{'accuracy': 0.866006600660066}	{'f1': 0.8652013668499585}
0.0074	18.0	5634	1.1296	{'accuracy': 0.863036303630363}	{'f1': 0.8615189712315606}
0.0074	19.0	5947	1.1099	{'accuracy': 0.8689768976897689}	{'f1': 0.867663744149239}
0.0046	20.0	6260	1.1250	{'accuracy': 0.866996699669967}	{'f1': 0.8657113367537151}

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.1