metadata

language:
  - en
license: cc-by-sa-4.0
library_name: span-marker
tags:
  - span-marker
  - token-classification
  - ner
  - named-entity-recognition
  - generated_from_span_marker_trainer
datasets:
  - tomaarsen/ner-orgs
metrics:
  - precision
  - recall
  - f1
widget:
  - text: >-
      Today in Zhongnanhai, General Secretary of the Communist Party of China,
      President of the country and honorary President of China's Red Cross,
      Zemin Jiang met with representatives of the 6th National Member Congress
      of China's Red Cross, and expressed warm greetings to the 20 million
      hardworking members on behalf of the Central Committee of the Chinese
      Communist Party and State Council.
  - text: >-
      On April 20, 2017, MGM Television Studios, headed by Mark Burnett formed a
      partnership with McLane and Buss to produce and distribute new content
      across a number of media platforms.
  - text: 'Postponed: East Fife v Clydebank, St Johnstone v'
  - text: >-
      Prime contractor was Hughes Aircraft Company Electronics Division which
      developed the Tiamat with the assistance of the NACA.
  - text: >-
      After graduating from Auburn University with a degree in Engineering in
      1985, he went on to play inside linebacker for the Pittsburgh Steelers for
      four seasons.
pipeline_tag: token-classification
co2_eq_emissions:
  emissions: 248.1008753496152
  source: codecarbon
  training_type: fine-tuning
  on_cloud: false
  cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
  ram_total_size: 31.777088165283203
  hours_used: 1.766
  hardware_used: 1 x NVIDIA GeForce RTX 3090
base_model: bert-base-cased
model-index:
  - name: SpanMarker with bert-base-cased on FewNERD, CoNLL2003, and OntoNotes v5
    results:
      - task:
          type: token-classification
          name: Named Entity Recognition
        dataset:
          name: FewNERD, CoNLL2003, and OntoNotes v5
          type: tomaarsen/ner-orgs
          split: test
        metrics:
          - type: f1
            value: 0.7946954813359528
            name: F1
          - type: precision
            value: 0.7958325880879986
            name: Precision
          - type: recall
            value: 0.793561619404316
            name: Recall

SpanMarker with bert-base-cased on FewNERD, CoNLL2003, and OntoNotes v5

This is a SpanMarker model trained on the FewNERD, CoNLL2003, and OntoNotes v5 dataset that can be used for Named Entity Recognition. This SpanMarker model uses bert-base-cased as the underlying encoder.

Model Details

Model Description

Model Type: SpanMarker
Encoder: bert-base-cased
Maximum Sequence Length: 256 tokens
Maximum Entity Length: 8 words
Training Dataset: FewNERD, CoNLL2003, and OntoNotes v5
Language: en
License: cc-by-sa-4.0

Model Sources

Repository: SpanMarker on GitHub
Thesis: SpanMarker For Named Entity Recognition

Model Labels

Label	Examples
ORG	"Texas Chicken", "IAEA", "Church 's Chicken"

Evaluation

Metrics

Label	Precision	Recall	F1
all	0.7958	0.7936	0.7947
ORG	0.7958	0.7936	0.7947

Uses

Direct Use for Inference

from span_marker import SpanMarkerModel

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-bert-base-orgs")
# Run inference
entities = model.predict("Postponed: East Fife v Clydebank, St Johnstone v")

Downstream Use

You can finetune this model on your own dataset.

Click to expand

from span_marker import SpanMarkerModel, Trainer

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-bert-base-orgs")

# Specify a Dataset with "tokens" and "ner_tag" columns
dataset = load_dataset("conll2003") # For example CoNLL2003

# Initialize a Trainer using the pretrained model & dataset
trainer = Trainer(
    model=model,
    train_dataset=dataset["train"],
    eval_dataset=dataset["validation"],
)
trainer.train()
trainer.save_model("tomaarsen/span-marker-bert-base-orgs-finetuned")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Sentence length	1	23.5706	263
Entities per sentence	0	0.7865	39

Training Hyperparameters

learning_rate: 5e-05
train_batch_size: 32
eval_batch_size: 32
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 3

Training Results

Epoch	Step	Validation Loss	Validation Precision	Validation Recall	Validation F1	Validation Accuracy
0.7131	3000	0.0061	0.7978	0.7830	0.7904	0.9764
1.4262	6000	0.0059	0.8170	0.7843	0.8004	0.9774
2.1393	9000	0.0061	0.8221	0.7938	0.8077	0.9772
2.8524	12000	0.0062	0.8211	0.8003	0.8106	0.9780

Environmental Impact

Carbon emissions were measured using CodeCarbon.

Carbon Emitted: 0.248 kg of CO2
Hours Used: 1.766 hours

Training Hardware

On Cloud: No
GPU Model: 1 x NVIDIA GeForce RTX 3090
CPU Model: 13th Gen Intel(R) Core(TM) i7-13700K
RAM Size: 31.78 GB

Framework Versions

Python: 3.9.16
SpanMarker: 1.5.1.dev
Transformers: 4.30.0
PyTorch: 2.0.1+cu118
Datasets: 2.14.0
Tokenizers: 0.13.3

Citation

BibTeX

@software{Aarsen_SpanMarker,
    author = {Aarsen, Tom},
    license = {Apache-2.0},
    title = {{SpanMarker for Named Entity Recognition}},
    url = {https://github.com/tomaarsen/SpanMarkerNER}
}