Instructions to use mai-data-ai/ner-bert-base-cased-conll2003 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mai-data-ai/ner-bert-base-cased-conll2003 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="mai-data-ai/ner-bert-base-cased-conll2003")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("mai-data-ai/ner-bert-base-cased-conll2003") model = AutoModelForTokenClassification.from_pretrained("mai-data-ai/ner-bert-base-cased-conll2003", device_map="auto") - Notebooks
- Google Colab
- Kaggle
BERT-base-cased fine-tuned for Named Entity Recognition (CoNLL-2003)
This model is bert-base-cased fine-tuned on CoNLL-2003 to detect four entity types in English text: PER (person), ORG (organization), LOC (location) and MISC (miscellaneous). Labels follow the IOB2 scheme.
It was trained as part of an NER project that compares sequence-labeling architectures built from scratch (LSTM, BiLSTM, BiLSTM+CRF) against fine-tuned transformers.
Results (CoNLL-2003 test set, seqeval)
| Entity | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| LOC | 0.9287 | 0.9365 | 0.9325 | 1668 |
| MISC | 0.7889 | 0.8305 | 0.8092 | 702 |
| ORG | 0.8960 | 0.9133 | 0.9046 | 1661 |
| PER | 0.9588 | 0.9635 | 0.9611 | 1617 |
| micro avg | 0.9096 | 0.9242 | 0.9168 | 5648 |
Validation F1 (best epoch): 0.9528.
Comparison with the other models in the project
| Model | Precision | Recall | F1 |
|---|---|---|---|
| LSTM (GloVe + char-CNN) | 0.7922 | 0.8430 | 0.8168 |
| BiLSTM (GloVe + char-CNN) | 0.8830 | 0.8962 | 0.8896 |
| BiLSTM + CRF (GloVe + char-CNN) | 0.9000 | 0.8952 | 0.8976 |
| DistilBERT-base-cased | 0.8956 | 0.9008 | 0.8982 |
| BERT-base-cased (this model) | 0.9096 | 0.9242 | 0.9168 |
The CRF mainly improves structural validity: on the test set it produced 3 invalid I- transitions versus 75 for the BiLSTM without CRF, and fewer entity-boundary errors (89 vs. 136).
Training details
- Base model:
bert-base-cased - Data: CoNLL-2003 (14,041 train / 3,250 validation / 3,453 test sentences)
- Label alignment: the label is placed on the first sub-word of each word; remaining sub-words are ignored (-100) in the loss
- Epochs: 5, learning rate 3e-5, batch size 16, weight decay 0.01, fp16, seed 42
- Best checkpoint selected by validation F1 (epoch 5)
- Framework: Hugging Face Transformers
Trainer
Usage
import re
import torch
from transformers import AutoModelForTokenClassification, AutoTokenizer
name = "mai-data-ai/ner-bert-base-cased-conll2003"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForTokenClassification.from_pretrained(name).eval()
def predict_entities(text):
words = re.findall(r"\w+(?:[-'β]\w+)*|[^\w\s]", text)
enc = tok(words, is_split_into_words=True, return_tensors="pt")
with torch.no_grad():
pred = model(**enc).logits[0].argmax(-1).tolist()
seen, entities, cur = set(), [], None
for i, wid in enumerate(enc.word_ids(0)):
if wid is None or wid in seen:
continue # use the first sub-word of each word only
seen.add(wid)
label = model.config.id2label[pred[i]]
if label == "O":
cur = None
elif label.startswith("B-") or cur is None or cur["type"] != label[2:]:
cur = {"type": label[2:], "text": words[wid]}
entities.append(cur)
else:
cur["text"] += " " + words[wid]
return [(e["text"], e["type"]) for e in entities]
print(predict_entities("Satya Nadella announced that Microsoft will invest in Egypt."))
The model was trained with one label per word (on the first sub-word), so decode at word level as above. Generic pipeline("token-classification", aggregation_strategy="simple") can split a name into two entities when a middle sub-word is predicted as B-.
Limitations
- Case-sensitive. In a quick manual test, the fully lowercase sentence
washington signed the deal with amazon.produced no detected entities. The model relies on capitalization, so it is not suited to informal or lowercased text. - News-domain data. CoNLL-2003 is English Reuters news from 1996-1997, so performance is likely lower on other domains and on names or organizations that are rare or newer than the data.
- MISC is the weakest class (F1 0.81) because it is a heterogeneous category.
- English only. Arabic or mixed-language text is not supported.
- The scores come from a single training run with one seed, so differences of a few tenths of a point between models should not be treated as significant.
- CoNLL-2003 is built on Reuters news text; check the dataset's terms of use before any commercial use.
- Downloads last month
- 49
Model tree for mai-data-ai/ner-bert-base-cased-conll2003
Base model
google-bert/bert-base-casedDataset used to train mai-data-ai/ner-bert-base-cased-conll2003
Space using mai-data-ai/ner-bert-base-cased-conll2003 1
Evaluation results
- F1 (seqeval, micro) on CoNLL-2003test set self-reported0.917
- Precision (seqeval, micro) on CoNLL-2003test set self-reported0.910
- Recall (seqeval, micro) on CoNLL-2003test set self-reported0.924