Hugging Face's logo

tomaarsen
/
span-marker-bert-base-cross-ner

Token Classification
SpanMarker
PyTorch
TensorBoard
Safetensors
English
ner
named-entity-recognition
Eval Results
Model card Files Metrics Community
1
tomaarsen HF staff commited on
Commit
fe10e18
•
1 Parent(s): 75a3ab5

Improve README

Browse files
Files changed (1) hide show
  1. README.md +113 -1
README.md CHANGED
@@ -8,12 +8,85 @@ tags:
8
  - ner
9
  - named-entity-recognition
10
  pipeline_tag: token-classification
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
  ---
12
 
13
  # SpanMarker for Named Entity Recognition
14
 
15
- This is a [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) model that can be used for Named Entity Recognition. In particular, this SpanMarker model uses [bert-base-cased](https://huggingface.co/bert-base-cased) as the underlying encoder.
16
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
17
 
18
  ## Usage
19
 
@@ -35,3 +108,42 @@ entities = model.predict("Amelia Earhart flew her single engine Lockheed Vega 5B
35
  ```
36
 
37
  See the [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) repository for documentation and additional information on this library.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
  - ner
9
  - named-entity-recognition
10
  pipeline_tag: token-classification
11
+ model-index:
12
+ - name: SpanMarker w. bert-base-cased on CrossNER by Tom Aarsen
13
+ results:
14
+ - task:
15
+ type: token-classification
16
+ name: Named Entity Recognition
17
+ dataset:
18
+ type: P3ps/Cross_ner
19
+ name: CrossNER
20
+ split: test
21
+ revision: 7cecbbb3d2eb8c75c8571c53e5a5270cfd0c5a9e
22
+ metrics:
23
+ - type: f1
24
+ value: 0.8785
25
+ name: F1
26
+ - type: precision
27
+ value: 0.8825
28
+ name: Precision
29
+ - type: recall
30
+ value: 0.8746
31
+ name: Recall
32
+ datasets:
33
+ - P3ps/Cross_ner
34
+ language:
35
+ - en
36
+ metrics:
37
+ - f1
38
+ - recall
39
+ - precision
40
  ---
41
 
42
  # SpanMarker for Named Entity Recognition
43
 
44
+ This is a [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) model that can be used for Named Entity Recognition. In particular, this SpanMarker model uses [bert-base-cased](https://huggingface.co/bert-base-cased) as the underlying encoder. See [train.py](train.py) for the training script.
45
 
46
+ ## Labels & Metrics
47
+
48
+ | **Label** | **Examples** | **Precision** | **Recall** | **F1** |
49
+ |:-------------------|---|---------------:|-----------:|-------:|
50
+ | **all** | - | 88.25 | 87.46 | 87.85 |
51
+ | academicjournal | "New Journal of Physics", "EPL", "European Physical Journal B" | 84.04 | 96.34 | 89.77 |
52
+ | album | "Tellin' Stories", "Generation Terrorists", "Country Airs" | 90.71 | 85.81 | 88.19 |
53
+ | algorithm | "LDA", "PCA", "gradient descent" | 76.27 | 79.65 | 77.92 |
54
+ | astronomicalobject | "Earth", "Sun", "Halley's comet" | 92.00 | 93.24 | 92.62 |
55
+ | award | "Nobel Prize for Literature", "Acamedy Award for Best Actress", "Mandelbrot's awards" | 87.14 | 92.51 | 89.74 |
56
+ | band | "Clash", "Parliament Funkadelic", "Sly and the Family Stone" | 83.44 | 86.62 | 85.00 |
57
+ | book | "Nietzsche contra Wagner" , "Dionysian-Dithyrambs", "The Rebel" | 73.71 | 82.69 | 77.95 |
58
+ | chemicalcompound | "hydrogen sulfide", "Starch", "Lactic acid" | 71.21 | 71.21 | 71.21 |
59
+ | chemicalelement | "potassium", "Fluorine", "Chlorine" | 84.00 | 70.00 | 76.36 |
60
+ | conference | "SIGGRAPH", "IJCAI", "IEEE Transactions on Speech and Audio Processing" | 80.00 | 68.57 | 73.85 |
61
+ | country | "United Arab Emirates", "U.S.", "Canada" | 81.72 | 86.81 | 84.19 |
62
+ | discipline | "physics", "meteorology", "geography" | 48.39 | 55.56 | 51.72 |
63
+ | election | "2004 Canadian federal election", "2006 Canadian federal election", "1999 Scottish Parliament election" | 96.61 | 97.85 | 97.23 |
64
+ | enzyme | "RNA polymerase", "Phosphoinositide 3-kinase", "Protein kinase C" | 77.27 | 91.89 | 83.95 |
65
+ | event | "Cannes Film Festival", "2019 Special Olympics World Summer Games", "2017 Western Iraq campaign" | 75.00 | 66.30 | 70.38 |
66
+ | field | "computational imaging", "electronics", "information theory" | 89.80 | 83.02 | 86.27 |
67
+ | literarygenre | "novel", "satire", "short story" | 70.24 | 68.60 | 69.41 |
68
+ | location | "China", "BOMBAY", "Serbia" | 95.21 | 93.72 | 94.46 |
69
+ | magazine | "The Atlantic", "The American Spectator", "Astounding Science Fiction" | 81.48 | 78.57 | 80.00 |
70
+ | metrics | "BLEU", "precision", "DCG" | 72.53 | 81.48 | 76.74 |
71
+ | misc | "Serbian", "Belgian", "The Birth of a Nation" | 81.69 | 74.08 | 77.70 |
72
+ | musicalartist | "Chuck Burgi", "John Miceli", "John O'Reilly" | 79.67 | 87.11 | 83.23 |
73
+ | musicalinstrument | "koto", "bubens", "def" | 66.67 | 22.22 | 33.33 |
74
+ | musicgenre | "Christian rock", "Punk rock", "romantic melodicism" | 86.49 | 90.57 | 88.48 |
75
+ | organisation | "IRISH TIMES", "Comintern", "Wimbledon" | 91.37 | 90.85 | 91.11 |
76
+ | person | "Gong Zhichao", "Liu Lufung", "Margret Crowley" | 94.15 | 92.31 | 93.22 |
77
+ | poem | "Historia destructionis Troiae", "I Am Joaquin", "The Snow Man" | 83.33 | 68.63 | 75.27 |
78
+ | politicalparty | "New Democratic Party", "Bloc Québécois", "Liberal Party of Canada" | 87.50 | 90.17 | 88.82 |
79
+ | politician | "Susan Kadis", "Simon Strelchik", "Lloyd Helferty" | 86.16 | 88.93 | 87.52 |
80
+ | product | "AlphaGo", "WordNet", "Facial recognition system" | 60.82 | 70.24 | 65.19 |
81
+ | programlang | "R", "C++", "Java" | 92.00 | 71.88 | 80.70 |
82
+ | protein | "DNA methyltransferase", "tau protein", "Amyloid beta" | 60.29 | 59.42 | 59.85 |
83
+ | researcher | "Sirovich", "Kirby", "Matthew Turk" | 87.50 | 78.65 | 82.84 |
84
+ | scientist | "Matjaž Perc", "Cotton", "Singer" | 82.04 | 88.48 | 85.14 |
85
+ | song | "Right Where I'm Supposed to Be", "Easy", "Three Times a Lady" | 84.78 | 90.70 | 87.64 |
86
+ | task | "robot control", "elevator scheduling", "telecommunications" | 76.19 | 74.42 | 75.29 |
87
+ | theory | "Big Bang", "general theory of relativity", "Ptolemaic planetary theories" | 100.00 | 16.67 | 28.57 |
88
+ | university | "University of Göttingen", "Duke", "Imperial Academy of Sciences" | 77.14 | 91.01 | 83.51 |
89
+ | writer | "Thomas Mann", "George Bernard Shaw", "Thomas Hardy" | 76.29 | 82.84 | 79.43 |
90
 
91
  ## Usage
92
 
 
108
  ```
109
 
110
  See the [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) repository for documentation and additional information on this library.
111
+
112
+ ## Training procedure
113
+
114
+ ### Training hyperparameters
115
+
116
+ The following hyperparameters were used during training:
117
+ - learning_rate: 5e-05
118
+ - train_batch_size: 32
119
+ - eval_batch_size: 32
120
+ - seed: 42
121
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
122
+ - lr_scheduler_type: linear
123
+ - lr_scheduler_warmup_ratio: 0.1
124
+ - num_epochs: 3
125
+
126
+ ### Training results
127
+
128
+ | Training Loss | Epoch | Step | Validation Loss | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
129
+ |:-------------:|:-----:|:----:|:---------------:|:-----------------:|:--------------:|:----------:|:----------------:|
130
+ | 0.0521 | 0.25 | 200 | 0.0375 | 0.7149 | 0.6033 | 0.6544 | 0.8926 |
131
+ | 0.0225 | 0.5 | 400 | 0.0217 | 0.8001 | 0.7878 | 0.7939 | 0.9400 |
132
+ | 0.0189 | 0.75 | 600 | 0.0168 | 0.8526 | 0.8288 | 0.8405 | 0.9534 |
133
+ | 0.0157 | 1.01 | 800 | 0.0160 | 0.8481 | 0.8366 | 0.8423 | 0.9543 |
134
+ | 0.0116 | 1.26 | 1000 | 0.0158 | 0.8570 | 0.8568 | 0.8569 | 0.9582 |
135
+ | 0.0119 | 1.51 | 1200 | 0.0145 | 0.8752 | 0.8550 | 0.8650 | 0.9607 |
136
+ | 0.0102 | 1.76 | 1400 | 0.0145 | 0.8766 | 0.8555 | 0.8659 | 0.9601 |
137
+ | 0.01 | 2.01 | 1600 | 0.0139 | 0.8744 | 0.8718 | 0.8731 | 0.9629 |
138
+ | 0.0072 | 2.26 | 1800 | 0.0144 | 0.8748 | 0.8684 | 0.8716 | 0.9625 |
139
+ | 0.0066 | 2.51 | 2000 | 0.0140 | 0.8803 | 0.8738 | 0.8770 | 0.9645 |
140
+ | 0.007 | 2.76 | 2200 | 0.0138 | 0.8831 | 0.8739 | 0.8785 | 0.9644 |
141
+
142
+
143
+ ### Framework versions
144
+
145
+ - SpanMarker 1.2.4
146
+ - Transformers 4.31.0
147
+ - Pytorch 2.0.1+cu118
148
+ - Datasets 2.14.3
149
+ - Tokenizers 0.13.2