SpanMarker with roberta-large on FewNERD, CoNLL2003, and OntoNotes v5
This is a SpanMarker model trained on the FewNERD, CoNLL2003, and OntoNotes v5 dataset that can be used for Named Entity Recognition. This SpanMarker model uses roberta-large as the underlying encoder.
Model Details
Model Description
Model Sources
Model Labels
Label |
Examples |
ORG |
"IAEA", "Church 's Chicken", "Texas Chicken" |
Evaluation
Metrics
Label |
Precision |
Recall |
F1 |
ORG |
0.8238 |
0.7970 |
0.81019 |
Uses
Direct Use for Inference
from span_marker import SpanMarkerModel
model = SpanMarkerModel.from_pretrained("nbroad/span-marker-roberta-large-orgs-v1")
entities = model.predict("The program is classified in the National Collegiate Athletic Association (NCAA) Division I Bowl Subdivision (FBS), and the team competes in the Big 12 Conference.")
Downstream Use
You can finetune this model on your own dataset.
Click to expand
from span_marker import SpanMarkerModel, Trainer
model = SpanMarkerModel.from_pretrained("nbroad/span-marker-roberta-large-orgs-v1")
dataset = load_dataset("conll2003")
trainer = Trainer(
model=model,
train_dataset=dataset["train"],
eval_dataset=dataset["validation"],
)
trainer.train()
trainer.save_model("nbroad/span-marker-roberta-large-orgs-v1-finetuned")
Training Details
Training Set Metrics
Training set |
Min |
Median |
Max |
Sentence length |
1 |
23.5706 |
263 |
Entities per sentence |
0 |
0.7865 |
39 |
Training Hyperparameters
- learning_rate: 3e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.05
- num_epochs: 3
- mixed_precision_training: Native AMP
Training Results
Epoch |
Step |
Validation Loss |
Validation Precision |
Validation Recall |
Validation F1 |
Validation Accuracy |
0.1430 |
600 |
0.0085 |
0.7425 |
0.7383 |
0.7404 |
0.9726 |
0.2860 |
1200 |
0.0078 |
0.7503 |
0.7516 |
0.7510 |
0.9741 |
0.4290 |
1800 |
0.0077 |
0.6962 |
0.8107 |
0.7491 |
0.9718 |
0.5720 |
2400 |
0.0060 |
0.8074 |
0.7486 |
0.7769 |
0.9753 |
0.7150 |
3000 |
0.0057 |
0.8135 |
0.7717 |
0.7921 |
0.9770 |
0.8580 |
3600 |
0.0059 |
0.7997 |
0.7764 |
0.7879 |
0.9763 |
1.0010 |
4200 |
0.0057 |
0.7860 |
0.8051 |
0.7954 |
0.9771 |
1.1439 |
4800 |
0.0058 |
0.7907 |
0.7717 |
0.7811 |
0.9763 |
1.2869 |
5400 |
0.0058 |
0.8116 |
0.7803 |
0.7956 |
0.9774 |
1.4299 |
6000 |
0.0056 |
0.7918 |
0.7850 |
0.7884 |
0.9770 |
1.5729 |
6600 |
0.0056 |
0.8097 |
0.7837 |
0.7965 |
0.9769 |
1.7159 |
7200 |
0.0055 |
0.8113 |
0.7790 |
0.7948 |
0.9765 |
1.8589 |
7800 |
0.0052 |
0.8095 |
0.7970 |
0.8032 |
0.9782 |
2.0019 |
8400 |
0.0054 |
0.8244 |
0.7782 |
0.8006 |
0.9774 |
2.1449 |
9000 |
0.0053 |
0.8238 |
0.7970 |
0.8102 |
0.9782 |
2.2879 |
9600 |
0.0053 |
0.82 |
0.7901 |
0.8048 |
0.9773 |
2.4309 |
10200 |
0.0053 |
0.8243 |
0.7936 |
0.8086 |
0.9785 |
2.5739 |
10800 |
0.0053 |
0.8159 |
0.7953 |
0.8055 |
0.9781 |
2.7169 |
11400 |
0.0053 |
0.8072 |
0.8034 |
0.8053 |
0.9784 |
2.8599 |
12000 |
0.0052 |
0.8111 |
0.8017 |
0.8064 |
0.9782 |
Framework Versions
- Python: 3.10.12
- SpanMarker: 1.5.0
- Transformers: 4.35.2
- PyTorch: 2.1.0a0+32f93b1
- Datasets: 2.15.0
- Tokenizers: 0.15.0
Citation
BibTeX
@software{Aarsen_SpanMarker,
author = {Aarsen, Tom},
license = {Apache-2.0},
title = {{SpanMarker for Named Entity Recognition}},
url = {https://github.com/tomaarsen/SpanMarkerNER}
}