Hugging Face's logo

judithrosell
/
JNLPBA_bioBERT_NER

Token Classification
Transformers
TensorBoard
Safetensors
bert
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
judithrosell commited on
Commit
0ca83ca
•
1 Parent(s): 3635e25

End of training

Browse files
Files changed (1) hide show
  1. README.md +105 -0
README.md ADDED
@@ -0,0 +1,105 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: dmis-lab/biobert-v1.1
3
+ tags:
4
+ - generated_from_trainer
5
+ model-index:
6
+ - name: JNLPBA_bioBERT_NER
7
+ results: []
8
+ ---
9
+
10
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
11
+ should probably proofread and complete it, then remove this comment. -->
12
+
13
+ # JNLPBA_bioBERT_NER
14
+
15
+ This model is a fine-tuned version of [dmis-lab/biobert-v1.1](https://huggingface.co/dmis-lab/biobert-v1.1) on the None dataset.
16
+ It achieves the following results on the evaluation set:
17
+ - Loss: 0.1433
18
+ - Seqeval classification report: precision recall f1-score support
19
+
20
+ DNA 0.81 0.83 0.82 1593
21
+ RNA 0.79 0.83 0.81 1400
22
+ cell_line 0.77 0.82 0.79 1016
23
+ cell_type 0.98 0.96 0.97 37439
24
+ protein 0.85 0.86 0.85 2992
25
+
26
+ micro avg 0.95 0.94 0.95 44440
27
+ macro avg 0.84 0.86 0.85 44440
28
+ weighted avg 0.95 0.94 0.95 44440
29
+
30
+
31
+ ## Model description
32
+
33
+ More information needed
34
+
35
+ ## Intended uses & limitations
36
+
37
+ More information needed
38
+
39
+ ## Training and evaluation data
40
+
41
+ More information needed
42
+
43
+ ## Training procedure
44
+
45
+ ### Training hyperparameters
46
+
47
+ The following hyperparameters were used during training:
48
+ - learning_rate: 2e-05
49
+ - train_batch_size: 16
50
+ - eval_batch_size: 16
51
+ - seed: 42
52
+ - gradient_accumulation_steps: 2
53
+ - total_train_batch_size: 32
54
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
55
+ - lr_scheduler_type: linear
56
+ - num_epochs: 3
57
+
58
+ ### Training results
59
+
60
+ | Training Loss | Epoch | Step | Validation Loss | Seqeval classification report |
61
+ |:-------------:|:-----:|:----:|:---------------:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
62
+ | 0.2673 | 1.0 | 582 | 0.1492 | precision recall f1-score support
63
+
64
+ DNA 0.79 0.81 0.80 1593
65
+ RNA 0.78 0.79 0.79 1400
66
+ cell_line 0.74 0.84 0.79 1016
67
+ cell_type 0.98 0.96 0.97 37439
68
+ protein 0.84 0.86 0.85 2992
69
+
70
+ micro avg 0.95 0.94 0.94 44440
71
+ macro avg 0.83 0.85 0.84 44440
72
+ weighted avg 0.95 0.94 0.94 44440
73
+ |
74
+ | 0.1408 | 2.0 | 1164 | 0.1469 | precision recall f1-score support
75
+
76
+ DNA 0.83 0.80 0.81 1593
77
+ RNA 0.78 0.83 0.80 1400
78
+ cell_line 0.75 0.83 0.79 1016
79
+ cell_type 0.98 0.96 0.97 37439
80
+ protein 0.84 0.87 0.85 2992
81
+
82
+ micro avg 0.95 0.94 0.95 44440
83
+ macro avg 0.84 0.86 0.85 44440
84
+ weighted avg 0.95 0.94 0.95 44440
85
+ |
86
+ | 0.1237 | 3.0 | 1746 | 0.1433 | precision recall f1-score support
87
+
88
+ DNA 0.81 0.83 0.82 1593
89
+ RNA 0.79 0.83 0.81 1400
90
+ cell_line 0.77 0.82 0.79 1016
91
+ cell_type 0.98 0.96 0.97 37439
92
+ protein 0.85 0.86 0.85 2992
93
+
94
+ micro avg 0.95 0.94 0.95 44440
95
+ macro avg 0.84 0.86 0.85 44440
96
+ weighted avg 0.95 0.94 0.95 44440
97
+ |
98
+
99
+
100
+ ### Framework versions
101
+
102
+ - Transformers 4.35.2
103
+ - Pytorch 2.1.0+cu118
104
+ - Datasets 2.15.0
105
+ - Tokenizers 0.15.0