JNLPBA_bioBERT_NER

This model is a fine-tuned version of dmis-lab/biobert-v1.1 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.1445

• Seqeval classification report: precision recall f1-score support

     DNA       0.72      0.80      0.76       507
     RNA       0.81      0.83      0.82      1593
  

  cell_line 0.76 0.78 0.77 5750 cell_type 0.76 0.81 0.79 618 protein 0.81 0.81 0.81 1452

  micro avg 0.77 0.80 0.78 9920 macro avg 0.77 0.81 0.79 9920

weighted avg 0.77 0.80 0.78 9920

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Seqeval classification report
0.2687	1.0	582	0.1504	precision recall f1-score support

     DNA       0.72      0.81      0.76       507
     RNA       0.78      0.82      0.80      1593

cell_line 0.75 0.77 0.76 5750 cell_type 0.76 0.81 0.78 618 protein 0.80 0.81 0.80 1452

micro avg 0.76 0.79 0.77 9920 macro avg 0.76 0.80 0.78 9920 weighted avg 0.76 0.79 0.77 9920 | | 0.1412 | 2.0 | 1164 | 0.1461 | precision recall f1-score support

     DNA       0.72      0.81      0.76       507
     RNA       0.83      0.79      0.81      1593

cell_line 0.75 0.77 0.76 5750 cell_type 0.75 0.82 0.78 618 protein 0.85 0.75 0.80 1452

micro avg 0.78 0.78 0.78 9920 macro avg 0.78 0.79 0.78 9920 weighted avg 0.78 0.78 0.78 9920 | | 0.1251 | 3.0 | 1746 | 0.1445 | precision recall f1-score support

     DNA       0.72      0.80      0.76       507
     RNA       0.81      0.83      0.82      1593

cell_line 0.76 0.78 0.77 5750 cell_type 0.76 0.81 0.79 618 protein 0.81 0.81 0.81 1452

micro avg 0.77 0.80 0.78 9920 macro avg 0.77 0.81 0.79 9920 weighted avg 0.77 0.80 0.78 9920 |

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu118
• Datasets 2.15.0
• Tokenizers 0.15.0