README.md

metadata

license: other
license_name: elda-community-license-1.0
license_link: LICENSE
language:
  - ko
  - en
  - ja
library_name: transformers
pipeline_tag: token-classification
datasets:
  - klue/klue
  - AmazonScience/massive
tags:
  - korean
  - japanese
  - multilingual
  - intent-detection
  - speech-act
  - named-entity-recognition
  - span-extraction
  - conversational-ai
  - real-time
  - encoder
  - modernbert
  - byte-level-bpe
  - long-context
extra_gated_heading: Access to Intenter
extra_gated_description: >-
  Intenter is released for research, evaluation, internal validation and
  education. Tell us who you are and we will grant access.
extra_gated_prompt: >-
  By requesting access you agree to the Elda Community License 1.0: no
  commercial use, no redistribution of the weights or derivatives, and
  attribution as "Built with Elda". Commercial licensing is available on
  request.
extra_gated_fields:
  Name: text
  Email: text
  Affiliation: text
  Intended use: text
  I agree to the Elda Community License: checkbox

Elda-Intenter — real-time speech-act + entity extraction in a single pass

Korean-first, multilingual (ko · ja · en). 307M parameters · 8,192-token window · single forward pass.

⚠ Latency is not stated for this build. The previous release measured p50 27.8 ms on a 12-layer / 512-token backbone; this one is 22 layers with a 8,192 window, so that number does not carry over and we have not re-measured it.

One forward pass returns four aligned channels: what the speaker is doing (speech act), what they are talking about (entities), and the attribute/predicate structure binding them. Not a general-purpose NER model — the perception layer of a live conversational system.

"간판의 품격이라는 음식점이 있다며, 그 음식점은 어디에 있냐고"

  intent    NARRATE · DESCRIBE · QUESTION      (three spans, one pass)
  entity    간판의 품격 → ORGANIZATION
            음식점      → ORGANIZATION
  signals   is_pure_negation=0 · is_claim=0 · polarity=0 · depends_on_prev=1

---

What is different here — byte-level spans, so an unseen name is still a span

This build runs on a byte-level BPE tokenizer (Gemma-2 vocabulary, 256k). That is a decoder-style property in an encoder, and it is the point of the model, not a detail.

28 shop names that are not in any index, tokenized:
  previous build (SentencePiece unigram)   20 / 28 become [UNK]
  this build (byte-level BPE)               0 / 28 become [UNK]

A name that becomes [UNK] cannot be a span, and a missing span is worse than a wrong type: the layer downstream has no string to look up. With byte fallback every surface is representable — rare syllables, emoji, (@_!)--/ punctuation runs — so the span survives even when the name is new.

"🍜 너울사인 간판 재질 알려줘"   → 너울사인 / ORGANIZATION · REQUEST
"ㅸㅿ뷁공방 어디야"              → ㅸㅿ뷁공방 / ORGANIZATION · QUESTION

Both are shop names that do not exist; the second is deliberately unpronounceable. Neither produces an [UNK] token on this build.

★ Representable is not the same as emitted. On a held-out probe of synthetic shop names built from syllables the previous tokenizer could not encode, anchor reach went 0% → 95% in the question frame — but the same names inside an emoji-heavy frame still abstained 70–75% of the time. Tokenization removes the floor; the span still has to be taught. That gap is this model's open work.

---

Results

All sets below are excluded from training by construction.

MASSIVE (dev, re-annotated) — conversational speech, same five peers

Voice-assistant utterances, our actual traffic shape. Same axes, same particle rule; peers emit KLUE's six types, so only the shared axes are comparable and type is omitted.

	abstention ↓	span recall	boundary	precision	anchor reach
ko ours (469 sent.)	6.9	93.1	95.2	96.5	84.3
ko — 5 peers	33.7 – 67.9	32.1 – 66.3	69.9 – 86.4	89.4 – 94.7	6.6 – 72.7
ja ours (749 sent.)	6.9	93.1	97.3	95.5	83.4
ja — 5 peers	30.1 – 76.2	23.8 – 69.9	50.8 – 84.0	72.3 – 93.9	18.8 – 48.0
en ours (796 sent.)	8.5	91.5	96.6	84.5	89.9
en — 5 peers	38.6 – 69.4	30.6 – 61.4	12.5 – 63.3	90.7 – 94.8	28.5 – 64.0

We lead every axis on ko and ja, and all but precision on en. Entity F1 under our own scorer: ja 79.7 · ko 85.5 · en 74.5 (one trailing function word allowed).

Not comparable to published MASSIVE scores: that benchmark is intent classification plus slot filling with 55 slot types. We map 21 onto 11 of our entity types; peers are KLUE-NER models run outside their training domain. Zero training overlap on dev is verified by the evaluator.

KLUE-NER (dev · 600 sentences / 1,687 gold), entity-level F1

	entity F1	boundary F1
exact match	84.6	86.9
allowing one trailing Korean particle	85.7	88.0

Korean particles attach to the noun. The second row accepts one trailing particle when the start offset matches. ⚠ On this build the two rows are only 1.1 pp apart, because the model now emits bare nouns — see Korean particles under Limitations.

Measured with our own scorer on the same inputs, a KLUE-NER specialist fine-tuned on the full KLUE train set scores 83.4 (soddokayo/klue-roberta-large-klue-ner). This model sees 12,500 KLUE-NER training sentences (of 21,008) and is otherwise a general conversational model.

build	KLUE F1 (one particle)	note
previous release (12-layer, 512)	63.9	SentencePiece unigram
this release (22-layer, 8,192)	85.7	byte-level BPE + KLUE-NER training sentences

The jump is two changes at once — backbone and training data — so it is not a clean ablation of either.

KLUE-NER — the same five peers, on their own training domain

Every peer is a dedicated KLUE-NER model fine-tuned on all 21,008 KLUE-NER training sentences, emitting exactly KLUE's six types. We see 12,500 of them, emit 17 types projected onto those six, and run four channels plus six auxiliary heads in the same pass.

Re-measured on this build (v668) with the same five peers and the same scorer.

axis	ours	KF-DeBERTa	KLUE-BERT	RoBERTa-large	KoELECTRA	RoBERTa-base
type accuracy	96.7	98.2	97.7	98.1	95.1	92.2
abstention (lower better)	5.8	2.0	2.4	13.6	18.3	41.1
span recall	94.2	98.0	97.6	86.4	81.7	58.9
boundary	96.3	97.3	96.2	96.2	91.9	62.9
precision	96.4	97.2	97.4	98.0	96.0	96.7
anchor reach	87.9	94.8	94.6	69.8	54.8	13.0

We still lose every axis to the best peer, but the margins are now small: 2nd of six on boundary, 3rd on abstention, span recall and anchor reach, 4th on type, 5th on precision. Note the peer spread — anchor reach runs 13.0 to 94.8, so a single-peer table could have been picked to favour us.

Axes. abstention — nothing emitted at a gold position. span recall — something emitted there. boundary — returned span contains the gold string. precision — emitted spans landing on gold, over spans carrying a KLUE-mappable type (1,597/1,657); over all our spans it is 95.6, since some carry types KLUE does not annotate (WORK, ARTIFACTS, TERM, EVENT, DURATION). anchor reach — span returned under a type that opens a downstream lookup (LOCATION / ORGANIZATION; 447 of 1,687 gold here).

KLUE-NER news text is 19.9 % of our training mix (12,500 of 62,971 sentences); the rest is wiki, conversational Korean and synthetic templates. Whether these peers hold up on conversational Korean is untested here.

Reported by others, on their own runs — not our ruler, not verified by us: KLUE-RoBERTa-large 90.8 · XLM-R-large 85.9 · KR-BERT-base 77.2 · mBERT-base 73.2 (KLUE paper). Our re-scoring of a KLUE-NER RoBERTa-large on this split gave 83.4, not 90.8.

Frozen internal gates

Measured on this build (v668), per surface, not summed. Rows marked — were not re-run on this build; we leave them blank rather than carry an older build's number forward.

Role nouns — detection / typing (405 slots)	404 / 397
Referent expressions, held out (23 slots)	22 / 23 · typed 21
Safety floor — base model / serving form	12 / 12 · 14 / 14
Conversational-speech detection	—
Possessive structure — owner kept (40 surfaces)	37 / 40
Discourse signals — false positives on short utterances	—
Discourse signals — polarity grid	—
Formal-register interrogatives	12 / 14
Proper-name span probe, third-party frames (80 slots)	75 / 80
Name-span probe, third-party (698 slots)	—
Span-convention compliance (126 slots)	126 / 126

The proper-name probe uses frames supplied by a separate team and was measured here by us; the 698-slot name-span probe and span-convention compliance are run by that team on their own infrastructure (the former was not re-run on this build).

---

Why an encoder

	Elda-Intenter	typical small-LLM extraction
Parameters	307M	500M – 4B
Latency (single, incl. heads)	not measured on this build	hundreds of ms
Latency (batch of 16)	118.5 ms	seconds
Serving precision	fp32 (2 workers, 6.1 GB GPU)	varies
Output	fixed contract, char offsets	free text to be parsed

Measured on the production serving path (RTX 3060, fp32, heads attached). Span extraction is classification over token pairs, and a bidirectional encoder sees the whole utterance at once.

---

Output contract

Four Global Pointer channels over one shared mmBERT-base backbone (22 layers, 8,192-token window, byte-level BPE), plus binary signal heads and one span-attribute head. All channels are character spans over the original string.

channel	what it carries
intents	speech act over the utterance or clause — 18 types incl. AGREE / DISAGREE / QUESTION / COMMAND / NARRATE / DESCRIBE / REQUEST
entities	17 entity types × subtypes, plus about_speaker per span
attributes	modifiers bound to an entity
predicates	what is asserted of it
signals	four binary discourse signals, plus two three-valued judgements, each with probabilities

about_speaker is true / false / null (null = abstains). signals also carries place_q ("did the speaker point at one specific place or business?") and needs_map ("must something outside the model be consulted to answer?"), each "yes" / "no" / null with a probability. No threshold is applied to any of them — label and probability are both emitted and the cut is the consumer's. See ABOUT_SPEAKER_OUTPUT_CONTRACT.md and SIGNALS_JUDGE_OUTPUT_CONTRACT.md.

Spans may nest (「제 친구」 and 「친구」). Key by span offsets, not by surface.

Usage

No from_pretrained one-liner — custom four-channel span architecture.

import json, sys
from pathlib import Path

d = Path("path/to/this/repo")
sys.path.insert(0, str(d))
import modeling_btrack_4ch_w4 as M

m = M.load(str(d), "cuda")        # base + 5 gap-fill heads + about_speaker + W4 + 2 judge heads
print(json.dumps(M.infer(m, "판교 카카오 본사 어디야"), ensure_ascii=False, indent=1))

# model.safetensors carries the same weights in the same dtype, if you prefer it:
# from safetensors.torch import load_file
# m.load_state_dict(load_file(d / "model.safetensors"))

backbone_config.json is included — the repository is self-contained. md5sum -c MD5SUMS verifies a download in full; model.pt is bit-identical to the file answering live traffic.

Serve in fp32 (weights stored bf16). INT8 collapses this model — measured.

---

Limitations

• Korean first. ja and en are supported and measured; Korean gets the curated material.
• Not a general NER model. Type inventory and span scope come from a conversational product.
• Korean particles — this build trims them. Earlier releases kept the particle inside the span (어부가); this one emits the bare noun (어부). We measured the shift rather than intending it: the gap between exact match and one-particle-allowed scoring collapsed from 8.6 pp to 1.1 pp on this build, and on a held-out probe 79.5 % of the spans that changed were particle-bearing spans replaced by their bare form (97 matched pairs, zero cases of a noun being cut short). The training mix now includes KLUE-NER, whose gold excludes particles, and the model followed that convention. Whether to pull it back is an open decision for us — reverting costs KLUE score.
• News text is no longer the weak spot — 12,500 KLUE-NER news sentences are in the mix, and KLUE F1 went 63.9 → 85.7. That gain came with the losses listed under Known regressions below.
• is_pure_negation has false positives no threshold recovers (8 fp / 8 fn on 3,193 rows at τ=0.7). A soft signal, not a filter.
• Boundaries, not senses. Entity linking is a separate layer, not in this repository.

Known regressions (vs. the previous production model)

This build changed the backbone, so some axes moved. We publish the numbers rather than the summary.

gate	previous production	previous release	this build
risk-utterance floor, serving form (14 lines)	14 / 14	14 / 14	14 / 14
risk-utterance floor, base model (12 lines)	12 / 12	12 / 12	12 / 12
referential expressions, held out (23 slots)	23 / 23	23 / 23	22 / 23
connective-clause abstention (50 cells, lower is better)	0 / 50	0 / 50	2 / 50
role nouns, detected (405 slots)	401	404	404
role noun 「경비병」 typed as person (46 slots)	46	45	39
map-intent judge heads, consumer sheet (place_q · needs_map, of 115)	114 · 114	112 · 113	110 · 112
entity-set drift probe, net (boundary moves excluded)	baseline	−128	−65

The safety floor holds at production parity on both forms. What we gave up: one held-out referential slot, two connective-clause abstentions, and the role noun 「경비병」, which loses person typing in several of our probes at once — that is our open work. The drift probe roughly halved its loss against the previous release.

Latency — measured by the serving team on their GPU host: single request p50 28.7 → 30.5 ms, 16-request batch 126.5 → 114.3 ms against the previous production model. Not re-measured on other hardware.

---

This repository

Build 45b7cc4c. Refreshed roughly monthly. This build was promoted to production on 2026-10-06. Production has since moved to the same weights with two updated auxiliary heads (re-issued map-intent judge heads and a type-cover head, 2026-10-07); those heads are not in this repository. Two output contracts ship alongside the weights (ABOUT_SPEAKER_OUTPUT_CONTRACT.md, SIGNALS_JUDGE_OUTPUT_CONTRACT.md).

Attribution — third-party training data

Parts of the training corpus are adapted from publicly licensed datasets. Their licenses apply to those parts and are reproduced here.

KLUE — KLUE: Korean Language Understanding Evaluation, Park et al., 2021. Licensed under CC BY-SA 4.0. Source: https://github.com/KLUE-benchmark/KLUE · paper: arXiv:2105.09680.

Changes we made (this is an adaptation, not a copy). 12,500 sentences from the KLUE-NER training split were re-annotated under our own span convention and type inventory: KLUE's six entity types were mapped onto our seventeen, sentences overlapping our held-out probes were removed, and in 192 sentences common-noun person words that KLUE leaves unannotated (e.g. 사람) were labeled as persons under our convention. Intent, attribute and predicate channels carry no KLUE labels and are masked out during training. The dev split is used for evaluation only and appears nowhere in training — the evaluator verifies this and refuses to run otherwise.

MASSIVE — MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset, FitzGerald et al., Amazon Science. Licensed under CC BY 4.0. Source: https://github.com/alexa/massive.

Changes we made. Korean and Japanese rows were re-annotated under our span convention; 21 of MASSIVE's 55 slot types were mapped onto 11 of our entity types. MASSIVE's intent labels are not used — that channel is masked out per source. Dev rows used for evaluation are excluded from training by construction.

Everything else in the corpus is our own work. Training data and evaluation sets are not distributed with this repository.

License & access

Released under the Elda Community License 1.0 (see LICENSE).

• ✅ Research, evaluation, internal validation, education — free of charge
• ✳ Attribution: "Built with Elda"
• ⛔ Commercial use and redistribution require a separate agreement

Access is gated: tell us who you are and access is granted automatically. Training data and evaluation sets are not distributed.

Citation

@software{intenter2026,
  title  = {Elda-Intenter: real-time multilingual speech-act and entity extraction},
  author = {Elda AI},
  year   = {2026},
  url    = {https://huggingface.co/Elda-AI/intenter}
}