nhankins/multilingual_euph_lora

distilbert-base-multilingual-cased-lora-text-classification

This model is a fine-tuned version of distilbert/distilbert-base-multilingual-cased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5321
• Precision: 0.7883
• Recall: 0.8589
• F1 and accuracy: {'accuracy': 0.7487113402061856, 'f1': 0.8220802919708029}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1 and accuracy
0.6034	1.0	1552	0.5999	0.6781	0.9981	{'accuracy': 0.678479381443299, 'f1': 0.8075588121866564}
0.5756	2.0	3104	0.5892	0.7067	0.9418	{'accuracy': 0.696520618556701, 'f1': 0.8075194115243155}
0.5607	3.0	4656	0.5630	0.7449	0.8770	{'accuracy': 0.7139175257731959, 'f1': 0.8056042031523644}
0.5458	4.0	6208	0.5549	0.7544	0.8990	{'accuracy': 0.7338917525773195, 'f1': 0.8203566768160069}
0.5342	5.0	7760	0.5816	0.7381	0.9457	{'accuracy': 0.7364690721649485, 'f1': 0.8290848307563727}
0.5266	6.0	9312	0.5399	0.7705	0.8799	{'accuracy': 0.7416237113402062, 'f1': 0.8215398308856252}
0.519	7.0	10864	0.5315	0.7932	0.8408	{'accuracy': 0.7442010309278351, 'f1': 0.8162887552059231}
0.4878	8.0	12416	0.5318	0.7880	0.8541	{'accuracy': 0.7461340206185567, 'f1': 0.8197621225983532}
0.485	9.0	13968	0.5332	0.7851	0.8637	{'accuracy': 0.7480670103092784, 'f1': 0.8225147526100772}
0.5044	10.0	15520	0.5321	0.7883	0.8589	{'accuracy': 0.7487113402061856, 'f1': 0.8220802919708029}

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.17.0
• Tokenizers 0.15.2