Edit model card

GerMedBert_ATTR_V02_BRONCO

This model is a fine-tuned version of GerMedBERT/medbert-512 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0593
  • F1 Score: 0.8187
  • Precision: 0.8235
  • Recall: 0.8140
  • Accuracy: 0.8993
  • Num Input Tokens Seen: 4204246

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 6

Training results

Training Loss Epoch Step Validation Loss F1 Score Precision Recall Accuracy Input Tokens Seen
No log 0.25 81 0.1949 0.0 1.0 0.0 0.7240 175856
0.206 0.5 162 0.1196 0.4123 0.8393 0.2733 0.7847 349280
0.206 0.75 243 0.0945 0.6296 0.8673 0.4942 0.8351 526096
0.1019 1.0 324 0.0792 0.7584 0.8 0.7209 0.8698 702407
0.1019 1.25 405 0.0697 0.7874 0.7784 0.7965 0.8785 877047
0.0715 1.5 486 0.0697 0.7778 0.7447 0.8140 0.8715 1052647
0.0715 1.75 567 0.0673 0.7568 0.7826 0.7326 0.8715 1227287
0.0638 2.0 648 0.0680 0.7781 0.7358 0.8256 0.8715 1400330
0.0638 2.25 729 0.0622 0.7965 0.8084 0.7849 0.8941 1577786
0.0445 2.5 810 0.0593 0.8012 0.7943 0.8081 0.8906 1751722
0.0445 2.75 891 0.0583 0.8023 0.8023 0.8023 0.8906 1927258
0.0396 3.0 972 0.0579 0.8161 0.8068 0.8256 0.8993 2100749
0.0396 3.25 1053 0.0598 0.8125 0.7944 0.8314 0.8941 2276989
0.0289 3.5 1134 0.0592 0.8036 0.8232 0.7849 0.8941 2451501
0.0289 3.75 1215 0.0585 0.7954 0.7886 0.8023 0.8906 2628573
0.0271 4.0 1296 0.0571 0.8171 0.8034 0.8314 0.8993 2802576
0.0271 4.25 1377 0.0581 0.8235 0.8333 0.8140 0.9045 2978368
0.0194 4.5 1458 0.0619 0.7978 0.7717 0.8256 0.8837 3154544
0.0194 4.75 1539 0.0612 0.8048 0.8323 0.7791 0.8958 3330400
0.0193 5.0 1620 0.0585 0.8059 0.8155 0.7965 0.8958 3505555
0.0193 5.25 1701 0.0587 0.8187 0.8235 0.8140 0.9010 3680771
0.0153 5.5 1782 0.0592 0.8242 0.8171 0.8314 0.9010 3856947
0.0153 5.75 1863 0.0592 0.8163 0.8187 0.8140 0.8993 4030371
0.0146 6.0 1944 0.0593 0.8187 0.8235 0.8140 0.8993 4204246

Framework versions

  • Transformers 4.40.1
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.0
  • Tokenizers 0.19.1
Downloads last month
9
Safetensors
Model size
109M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for BachelorThesis/GerMedBert_ATTR_V02_BRONCO

Finetuned
(4)
this model

Dataset used to train BachelorThesis/GerMedBert_ATTR_V02_BRONCO

Collection including BachelorThesis/GerMedBert_ATTR_V02_BRONCO