impresso-project/ner-hipe2020-mmbert-base-final

Impresso HIPE-2022 NER model

Named-entity recognition for historical newspapers: pers, loc, org, prod, time, in French and German. This is the model behind the entity annotations of the Impresso corpus, and it ships with a ready-to-use Hugging Face pipeline (impresso-ner): plain text in, entities with character offsets out.

Use it

# pip install transformers torch nltk
from transformers import pipeline

ner = pipeline("impresso-ner", model="impresso-project/ner-hipe2020-mmbert-base-final",
               revision="impresso-ner-v1", trust_remote_code=True)
ner("In 1789, King Louis XVI convened the Estates-General at the Palace of Versailles.")
# [{'type': 'pers', 'surface': 'King Louis XVI', 'lOffset': 9, 'rOffset': 23,
#   'confidence_ner': ..., 'confidence_ner_entity': ..., 'confidence_ner_type': ...}, ...]
ner([text1, text2])   # one list per text

# Each string is one sentence by default. For longer texts:
ner(article, split_sentences=True)        # split with NLTK Punkt
ner(article, spans=[(0, 42), (43, 97)])   # your own sentence (start, end) spans
ner([a1, a2], spans=[spans1, spans2])     # one span list per text

• Pin the revision. impresso-ner-v1 is a tag: the code and weights it names never change. Requires transformers ≥ 5. trust_remote_code=True runs the two Python files of that revision (hf_pipeline.py, process.py); you can read them in this repository.
• Output. One dict per entity: type, surface, lOffset / rOffset (character offsets into your text, text[lOffset:rOffset] == surface), confidence_ner and its two parts confidence_ner_entity (is this an entity?) and confidence_ner_type (is it this type?). The output feeds the impresso-nel pipeline (impresso-project/nel-mgenre-multilingual, revision impresso-nel-v1) directly: nel({"text": text, "entities": ner(text)}).
• Sentences. The model reads one sentence at a time, truncated at 512 tokens (a warning is logged when that happens). Each string is one sentence unless you pass split_sentences=True (NLTK Punkt, German model plus abbreviations common in historical newspapers) or spans. Offsets always point into the string you passed. Entities never cross a sentence boundary.
• How it works. Each sentence's entities go through the Impresso post-processing (boundary clean-up, nested-entity removal). This is the code of the Impresso batch pipeline.
• Precision. fp32 by default. Passing dtype=torch.bfloat16 works but changes confidences and, rarely, entities.
• Without the pipeline. The model is a stock ModernBertForTokenClassification: AutoModelForTokenClassification.from_pretrained(...) loads it without remote code.

Model

Token-classification model fine-tuned from jhu-clsp/mmBERT-base on the impresso-project/ner-augmentation dataset (HIPE-2022, NE-COARSE-LIT, IOB2).

• Languages: fr, de
• Base model: jhu-clsp/mmBERT-base
• Training data: impresso-project/ner-augmentation
• Label scheme: IOB2 over NE-COARSE-LIT (pers / org / loc / prod / time). Single head: no fine-grained types, no components.

Test-set results (seqeval, entity-level)

	F1	fuzzy F1
HIPE-2020 de + fr	0.8149	0.8910
HIPE-2020 fr	0.8269	0.9021
HIPE-2020 de	0.7970	0.8752
HIPE-2020 en (not trained on)	0.6154	0.7705
HIPE-2020 all	0.7867	0.8736

All metrics

metric	value
hipe2020/de-fr_f1	0.8149
hipe2020/de-fr_fuzzy_f1	0.8910
hipe2020/de-fr_fuzzy_loc_f1	0.9269
hipe2020/de-fr_fuzzy_org_f1	0.6654
hipe2020/de-fr_fuzzy_pers_f1	0.9202
hipe2020/de-fr_fuzzy_prod_f1	0.7455
hipe2020/de-fr_fuzzy_time_f1	0.9124
hipe2020/de-fr_loc_f1	0.8830
hipe2020/de-fr_org_f1	0.6098
hipe2020/de-fr_pers_f1	0.7970
hipe2020/de-fr_precision	0.8229
hipe2020/de-fr_prod_f1	0.6140
hipe2020/de-fr_recall	0.8071
hipe2020/de-fr_time_f1	0.6923
hipe2020/de_f1	0.7970
hipe2020/de_fuzzy_f1	0.8752
hipe2020/de_fuzzy_loc_f1	0.9259
hipe2020/de_fuzzy_org_f1	0.6357
hipe2020/de_fuzzy_pers_f1	0.8961
hipe2020/de_fuzzy_prod_f1	0.6990
hipe2020/de_fuzzy_time_f1	0.9592
hipe2020/de_loc_f1	0.8862
hipe2020/de_org_f1	0.5645
hipe2020/de_pers_f1	0.7628
hipe2020/de_prod_f1	0.5545
hipe2020/de_time_f1	0.7789
hipe2020/en_f1	0.6154
hipe2020/en_fuzzy_f1	0.7705
hipe2020/en_fuzzy_loc_f1	0.8156
hipe2020/en_fuzzy_org_f1	0.5772
hipe2020/en_fuzzy_pers_f1	0.8503
hipe2020/en_fuzzy_prod_f1	0.6429
hipe2020/en_fuzzy_time_f1	0.7805
hipe2020/en_loc_f1	0.7244
hipe2020/en_org_f1	0.3077
hipe2020/en_pers_f1	0.6625
hipe2020/en_prod_f1	0.5926
hipe2020/en_time_f1	0.2051
hipe2020/fr_f1	0.8269
hipe2020/fr_fuzzy_f1	0.9021
hipe2020/fr_fuzzy_loc_f1	0.9276
hipe2020/fr_fuzzy_org_f1	0.6950
hipe2020/fr_fuzzy_pers_f1	0.9356
hipe2020/fr_fuzzy_prod_f1	0.7863
hipe2020/fr_fuzzy_time_f1	0.8739
hipe2020/fr_loc_f1	0.8797
hipe2020/fr_org_f1	0.6557
hipe2020/fr_pers_f1	0.8186
hipe2020/fr_prod_f1	0.6667
hipe2020/fr_time_f1	0.6195
hipe2020_f1	0.7867
hipe2020_fuzzy_f1	0.8736
hipe2020_fuzzy_loc_f1	0.9140
hipe2020_fuzzy_org_f1	0.6456
hipe2020_fuzzy_pers_f1	0.9085
hipe2020_fuzzy_prod_f1	0.7339
hipe2020_fuzzy_time_f1	0.8915
hipe2020_loc_f1	0.8647
hipe2020_org_f1	0.5466
hipe2020_pers_f1	0.7747
hipe2020_prod_f1	0.6116
hipe2020_time_f1	0.6154

Versions

Revision	What
impresso-ner-v1 (tag)	these weights + the impresso-ner pipeline
8d95f6dbe34d889f67745cc132013cc8372c0adb	these weights as first published (2026-07-23); the Impresso batch pipeline pins it
retrain-fr-de-en (branch)	a later retrain on fr, de and en (test F1 0.7964 in its card), published on main on 2026-09-14 and moved to this branch; not used by Impresso

Licences

The weights are under the licence in this card's header (CC BY-NC-SA 4.0, from the HIPE data): non-commercial use. The pipeline code is AGPL-3.0-or-later (see LICENSE).

The two code files (hf_pipeline.py, process.py) are copied unchanged from the Impresso impresso-inference repository, where process.py is the processing code the Impresso batch pipeline runs; do not edit them here.