ivlcic/snerta-12l-base

Snerta-12l Base

Snerta-12l Base is a multilingual named entity recognition model for 12 Slavic and neighbouring languages. It is a 278M-parameter microsoft/mdeberta-v3-base model fine-tuned on the complete, deduplicated multilingual training pool prepared for the paper When Is Multilingual Transfer Beneficial for Slavic Named Entity Recognition?

The model recognises four coarse entity types in IOB2 format:

Type	Meaning
PER	Person
LOC	Location
ORG	Organization
MISC	Miscellaneous entity, including source-specific labels mapped into the shared category

The underlying nine labels are O, B-LOC, I-LOC, B-MISC, I-MISC, B-ORG, I-ORG, B-PER, and I-PER.

Intended use

The model is intended for general-purpose named entity extraction from text in Bulgarian, Bosnian, Czech, Croatian, Macedonian, Polish, Russian, Slovak, Slovene, Albanian, Serbian, and Ukrainian. It is particularly useful when one shared model is preferable to maintaining a separate model for every language.

It is suitable as a component in search, document enrichment, media monitoring, corpus analysis, and entity-centric analytics. It should not be used as the sole mechanism for anonymisation, access control, legal decisions, or other high-stakes applications. Validate it on the target domain and language before deployment.

Usage

Install Transformers and create a token-classification pipeline with simple entity aggregation:

from transformers import pipeline

model_id = "ivlcic/snerta-12l-base"
ner = pipeline(
    task="token-classification",
    model=model_id,
    tokenizer=model_id,
    aggregation_strategy="simple",
)

text = (
    "Janez Novak... Metka Kralj,,. in Boris A. Novak živijo v Ljubljani "
    "in delajo za Microsoft."
)

for entity in ner(text):
    span = text[entity["start"]:entity["end"]]
    print(entity["entity_group"], span, round(float(entity["score"]), 3))

The example identifies Janez Novak, Metka Kralj, and Boris A. Novak as PER, Ljubljani as LOC, and Microsoft as ORG.

For applications that need to preserve an existing tokenisation—including whitespace and punctuation—the model can also be called with pre-split tokens:

import re

tokens = re.findall(r"\s+|\w+|[^\w\s]", text, flags=re.UNICODE)
entities = ner(
    tokens,
    is_split_into_words=True,
    delimiter="",
)[0]

for entity in entities:
    span = text[entity["start"]:entity["end"]]
    print(entity["entity_group"], span)

Inputs longer than 256 model tokens should be split into overlapping windows by the application. Take care to reconcile entities that cross window boundaries.

Training data

Source corpora belonging to the same language key were pooled before an 80/10/10 sentence-level train/validation/test split.

Language	Code	Source corpus or corpora	Prepared sentences before splitting and deduplication	Role in the paper
Bulgarian	bg	BSNLP / SlavNER	18,333	Main
Bosnian	bs	WikiANN / PAN-X	18,810	Lower-confidence auxiliary
Czech	cs	BSNLP / SlavNER; CNEC 2.0	20,864	Main
Croatian	hr	hr500k 1.0	24,794	Main
Macedonian	mk	WikiANN / PAN-X	16,227	Lower-confidence auxiliary
Polish	pl	BSNLP / SlavNER	20,423	Main
Russian	ru	BSNLP / SlavNER	25,141	Main
Slovak	sk	WikiANN / PAN-X	50,907	Lower-confidence auxiliary
Slovene	sl	BSNLP / SlavNER; ssj500k 2.3, ELEXIS-WSD 1.0, and SentiCoref 1.0 from SUK 1.0	48,701	Main
Albanian	sq	WikiANN / PAN-X	10,098	Lower-confidence auxiliary
Serbian	sr	SETimes.SR 1.0	3,891	Main
Ukrainian	uk	BSNLP / SlavNER; NER-UK 2.0	24,424	Main

The checkpoint uses the full, unbalanced training pool for all 12 language keys. The separate Croatian WikiANN data key supplied for the paper's source-quality ablation was not used to train this model.

After deduplication, the model's 12-language pool contains 190,590 training sentences, 25,796 validation sentences, and 26,979 test sentences. The training split contains approximately 3.27 million word-level tokens. The training pipeline labels only the first model subtoken of each original word and ignores later subtokens when computing the loss and metrics.

Deduplication and relationship to prior work

Unlike the precursor ivlcic/sour-sarma, this model was trained on sentence-deduplicated data. Duplicate matching was performed independently within each language data key on the NFKC-normalized, case-folded token sequence. When a duplicate crossed splits, the test occurrence was retained over validation and training, and validation was retained over training. Repetitions within a split were also removed.

Across the 12 language keys used for this checkpoint, 39,248 of 282,613 prepared sentence instances were removed. The supplementary audit reports 48,225 removals out of 331,498 instances when the separate Croatian WikiANN ablation source is also counted. Deduplication prevents exact sentence leakage but does not remove paraphrases, near-duplicates, or related sentences from the same document.

There is also an intentional label-space difference from the paper experiments. The controlled experiments in the paper use only PER, LOC, and ORG, mapping unsupported labels—including MISC—to O. This release retains MISC because it is intended for a broader operational usage scenario. Consequently, the benchmark below should not be compared directly with the paper's three-type, eight-evaluation-language, three-seed macro averages.

Source data terms

The combined corpus does not have a single uniform data license. Each source retains its own license and citation requirements. In particular, SUK 1.0 is distributed under CC BY-NC-SA 4.0, while hr500k 1.0 and SETimes.SR 1.0 are distributed under CC BY-SA 4.0. BSNLP / SlavNER, WikiANN / PAN-X, CNEC 2.0, and NER-UK 2.0 are governed by their respective source terms. The Apache-2.0 license in this model repository does not replace the source dataset terms.

Training procedure

The model was initialised from microsoft/mdeberta-v3-base and trained in FP32 with the following configuration:

Hyperparameter	Value
Maximum sequence length	256
Epochs	20
Per-device training batch size	16
Gradient accumulation steps	2
Effective training batch size	32
Per-device evaluation batch size	32
Optimizer	PyTorch AdamW
Learning rate	2e-5
Learning-rate schedule	Linear
Warmup ratio	0.06
Weight decay	0.01
Maximum gradient norm	1.0
Dropout	0.10
Random seed	2611
Checkpoint selection	Best validation entity-level F1

The run evaluated and saved a checkpoint after every epoch. The selected checkpoint was from epoch 19, with validation F1 92.60. Training completed 119,140 optimiser steps; the recorded training runtime was approximately 5 hours 58 minutes. The run used PyTorch 2.11.0, Transformers 5.8.0, and seqeval 1.2.2.

The learning rate and dropout follow the mDeBERTa-v3 setting selected by the paper's bounded validation sweep. In the Transformers mDeBERTa implementation, the configured dropout maps to hidden dropout and therefore affects the encoder layers as well as the token-classification head.

Evaluation

The released checkpoint was evaluated once on the combined deduplicated test split for all 12 language keys. Metrics were computed with seqeval at the entity level: a prediction is correct only when both its span and entity type match exactly. Precision, recall, and F1 below are micro-averaged over entities; accuracy is calculated over labeled word positions.

Entity type	Precision	Recall	F1
Overall	92.21	93.54	92.87
LOC	94.97	95.51	95.24
MISC	82.97	85.39	84.16
ORG	90.26	92.10	91.17
PER	93.78	95.12	94.44

Token accuracy is 98.62% and test loss is 0.1317. These values come from the saved CaNNopy evaluation artefact for this checkpoint.

Limitations and biases

• The benchmark uses a random sentence-level split of pooled source corpora. It does not measure cross-document, temporal, topical, or cross-domain generalisation.
• Results come from one training seed and are aggregated across all 12 languages. Performance can vary substantially by language and domain, and the aggregate score can obscure lower-performing subsets.
• The corpora differ in domain, segmentation, annotation density, provenance, and label definitions. MISC is especially heterogeneous across sources.
• Bosnian, Macedonian, Slovak, and Albanian supervision comes from automatically constructed WikiANN / PAN-X data and is considered lower confidence than the manually curated sources.
• The full-pool training recipe is intentionally unbalanced, so larger language corpora contribute more training examples.
• Exact sentence deduplication does not remove near-duplicates or guarantee document-level separation.
• Names, organisations, locations, and demographic groups that are rare or absent in the training corpora may be recognised less reliably. False positives and false negatives should be expected.
• Although the base encoder is multilingual, this fine-tuned model was trained and evaluated only on the 12 listed languages. Performance on other languages is unknown.
• Sequences are truncated at 256 model tokens unless the calling application implements windowing.

Reproducibility

• Training and evaluation code: ivlcic/cannopy
• Base checkpoint: microsoft/mdeberta-v3-base
• Precursor model: ivlcic/sour-sarma