---
language:
- hu
license: apache-2.0
base_model: SZTAKI-HLT/hubert-base-cc
pipeline_tag: token-classification
tags:
- hungarian
- ner
- corruption
datasets:
- K-Monitor/kmdb_base
metrics:
- precision
- recall
- f1
---

# kmdb_ner_v5

Hungarian named entity recognition for the
[K-Monitor press database](https://adatbazis.k-monitor.hu/). It does two jobs
at once: it finds people, organisations and places in an article, and it marks
which of them are the ones an editor would actually record for that article.

Labels are `B-/I-` + `{POS,NEG}_{PER,ORG,LOC}`. `POS` means the entity is
relevant to the article's case and belongs in the database; `NEG` means it is
merely mentioned.

Finetuned from `SZTAKI-HLT/hubert-base-cc` on 58,524 articles from
[`K-Monitor/kmdb_base`](https://huggingface.co/datasets/K-Monitor/kmdb_base):
spaCy (`hu_core_news_lg`) supplies the entity spans, and an entity is labelled
`POS` when it matches one of the names an editor tagged on that article.

## Results

Entity-level seqeval scores on 4,161 held-out articles. The honest comparison
against the previous `boapps/kmdb_ner_model` is the second column: that
checkpoint was trained with an unseeded split, so most of the general test set
was in its training data, whereas these articles were added to the database
after it was published.

| model | test F1 | F1 on articles the old model never saw |
| --- | --- | --- |
| `boapps/kmdb_ner_model` | 0.8244\* | 0.6684 |
| **`kmdb_ner_v5`** | 0.8171 | **0.7299** |

\* inflated by train/test leakage in the earlier checkpoint.

Per class, on those unseen articles:

| label | precision | recall | F1 | previous F1 |
| --- | --- | --- | --- | --- |
| NEG_LOC | 0.7817 | 0.7075 | 0.7427 | 0.7208 |
| NEG_ORG | 0.7530 | 0.7587 | 0.7558 | 0.7176 |
| NEG_PER | 0.7298 | 0.6752 | 0.7014 | 0.6505 |
| POS_LOC | 0.6219 | 0.7128 | 0.6642 | 0.5729 |
| POS_ORG | 0.6469 | 0.7130 | 0.6784 | 0.5270 |
| POS_PER | 0.6971 | 0.8029 | 0.7463 | 0.6484 |
| **micro avg** | 0.7242 | 0.7356 | **0.7299** | 0.6684 |

The largest gain is on `POS_ORG` (0.5270 → 0.6784): the previous training data
matched gold names by exact substring, so tagged organisations whose database
name carries a suffix — `Hunguest Hotels Zrt.`, `Honvédelmi Minisztérium (HM)` —
were labelled `NEG` and the model learned to ignore them.

## Usage

```python
from transformers import AutoTokenizer, pipeline

classifier = pipeline(
    "ner",
    model="K-Monitor/kmdb_ner_v5",
    aggregation_strategy="first",
    stride=8,
    tokenizer=AutoTokenizer.from_pretrained(
        "SZTAKI-HLT/hubert-base-cc", model_max_length=512
    ),
)
classifier(article_text)
```

Adjacent fragments of one entity have to be merged afterwards (huBERT splits
detached Hungarian suffixes, e.g. `Budapest` + `-en`); when merging, keep the
entity type and treat the result as `POS` if either fragment is `POS`. The
reference implementation is `join_entities` in
[`sajtoadatbazis-automat`](https://github.com/k-monitor/sajtoadatbazis-automat)
(`webapp/auto_kmdb/processors/NERProcessor.py`).

## Limitations

The `POS`/`NEG` supervision is derived automatically by matching spaCy entities
against editor-tagged names, so it inherits spaCy's boundary errors and the
matching heuristic's mistakes; it is not hand-annotated. Scores on articles
from after the training window (~0.73 F1) are well below in-distribution scores
(~0.82), so expect the real-world quality to track the former.