| --- |
| language: |
| - en |
| license: cc-by-nc-4.0 |
| library_name: pytorch |
| pipeline_tag: token-classification |
| tags: |
| - streaming |
| - asr |
| - ner |
| - pii |
| - redaction |
| - intent |
| - dialogue-act |
| - mamba |
| --- |
| |
| # Arc Tagger (10M), v0.1.0 |
|
|
| Arc Tagger is a **text-only, streaming tagger** with about 9.8 M parameters. It reads the word stream of **any ASR** (Deepgram, Nemotron, |
| Whisper, our own ASR, or plain text). As each word arrives, it emits entity and number spans, PII spans for redaction, profanity and |
| toxicity, discourse acts, intents, safety flags, speaker roles, events (speaker change, turn yield, sports, legal and meeting events), |
| content state and emotion (experimental). |
|
|
| * Checkpoint: `AT10Mv2` step 88000. It scored best on all gold sets of its run. |
| * Architecture: a word channel (8k byte-level BPE, 2 causal local-attention layers) feeds a 6-layer Mamba-3 trunk. Global KV-memory |
| attention layers, domain FiLM and adapters condition the trunk. The read-outs are definition-keyed tag heads plus a v2 span head (BIO |
| and type per family), a span verifier and pooled utterance, speaker and item heads. The model is causal and runs one recurrent step |
| per word, so a revision re-runs only the revised words. |
| * Runs on CPU. Measured on one core (busy shared box, 1 thread): **engine about 16 ms per word p50, 25 ms p95**. A full |
| `push_words()` call, including span decoding and read-outs, takes about 24 ms p50 and 45 ms p95. |
|
|
| ## Tag families |
|
|
| 276 tags across 10 read-outs. Every tag name comes from the Whissle ontology r8 plus the extension `ext_cc_roles1`. |
|
|
| | family | tags (examples) | read-out | |
| |---|---|---| |
| | Entities | `entity.mention.{person,organization,team,place,event_name,legal_ref,term,...}` (10) | word spans | |
| | Numbers | `entity.number.{date,time_of_day,duration,count,money,percent,measure,score,clock,ordinal,id_pii,...}` (16), `entity.numeral` | word spans | |
| | PII / redaction | `safety.pii.{private_name,phone_number,email,street_address,card_account_number,gov_id,date_of_birth,url_handle,public_contact}` spans; `safety.pii.{request,leak}` per utterance | spans and utterance | |
| | Profanity / toxicity | `safety.profanity.{mild,strong}`, `safety.toxicity.*` (6) | spans and utterance | |
| | Safety | `safety.scam.*` (6), `safety.crisis.self_harm`, `safety.threat.escalation`, `safety.child.*`, `safety.solicitation.dating_romantic` | utterance | |
| | Discourse act | `discourse.act.*` (statement, question, command_request, ...), `discourse.answer`, `discourse.backchannel`, `discourse.assistant.*` | utterance | |
| | Intent | 26 cross-domain `intent.*` (request_information, schedule_arrange, complain_criticize, ...), 60 assistant `discourse.intent.*` (alarm_set, weather_query, ...) | utterance | |
| | Speaker role | `speaker.role.*` (19, including contact-centre agent and customer) | per speaker (needs `spk` ids on the words) | |
| | Events | `speaker.change`, `discourse.turn_yield`, `discourse.interruption`, `event.sports.*` (18), `event.legal.*`, `event.meeting.{decision,action_item}` | per word | |
| | Content state | `content.mode`, `content.subject`, `content.game_state`, `content.legal_phase`, `content.meeting_phase` | per word (frame) | |
| | Emotion (experimental) | `affect.emotion.*` (5), `affect.arousal.*` (3) | utterance; low support, mostly "neutral" | |
|
|
| `ontology/` holds the full definitions. `config.json["columns"]` lists every `tag@readout` column. |
|
|
| ## Quickstart |
|
|
| ```bash |
| pip install huggingface_hub |
| huggingface-cli download WhissleAI/arc-tagger --local-dir arc-tagger # private repo: huggingface-cli login first |
| cd arc-tagger |
| pip install -r requirements.txt # CPU torch is fine: pip install torch --index-url https://download.pytorch.org/whl/cpu |
| python example.py |
| ``` |
|
|
| ```python |
| import sys; sys.path.insert(0, "arc-tagger") # the package ships in the repo (arctag/) |
| from arctag import ArcTagger |
| |
| tg = ArcTagger.from_pretrained("arc-tagger") # a local dir, or a repo id: ArcTagger.from_pretrained("WhissleAI/arc-tagger") |
| out = tg.tag("Hi, this is Maria from Northwind Bank. Can I get your phone number?", domain="talk") |
| print(out["redacted_text"]); print(out["spans"]); print(out["utterances"]) |
| |
| # streaming (words as the ASR emits them; interim hypotheses can be revised) |
| st = tg.stream(domain="assistant") |
| st.push_words(["set", "an", "alarm", "for"], final=False) # interim |
| up = st.push_words(["set", "an", "alarm", "for", "seven", "thirty."], revise_from=0, final=True) |
| up = st.endpoint() # end of utterance -> utterance read-outs in up["utterances"] |
| res = st.finalize() # full result (same schema as tag()) |
| ``` |
|
|
| ### Input |
|
|
| `tag(x, domain="auto", tier="T0")` |
| * `x` can be a `str`, a list of words, or a list of word dicts `{"w": str, "t0": s, "t1": s, "conf": float, "spk": str|int, "final": bool}`. |
| Every field except `w` is optional. |
| * Utterances end after words ending in `. ? !`. In streaming, call `endpoint()` at the ASR's end of utterance or end of turn. |
|
|
| `domain` is one of `auto | talk | meeting | court | assistant | sports | news`. |
| * It conditions the model through domain FiLM and adapters, and picks the per-domain thresholds. |
| * `talk` covers contact centre, podcasts and chat. Inside `sports` and `court`, the specific league or court type is detected from |
| lexicon cues. |
| * `auto` runs the "unknown" domain, which is weaker. **Set the domain when you know it.** |
|
|
| `tier` |
| * `T0` (default; 0 words of lookahead; the thresholds were fitted on it) |
| * `T1` (at most 3 words of delay) |
| * `T2` (utterance end) |
|
|
| Streaming: `push_words(words, final=True, revise_from=None, endpoint=False)` |
| * `revise_from=k` replaces `words[k:]`. Use it for interim hypotheses such as Deepgram's `is_final=false` or a growing RNNT hypothesis. |
|
|
| ### Output (JSON) |
|
|
| ```jsonc |
| { |
| "meta": {"model": "arc-tagger", "version": "0.1.0", "step": 88000, "domain": "talk", "fine_domain": "talk", "tier": "T0", "thresholds": "..."}, |
| "words": ["Hi,", "this", "is", "Maria", ...], |
| "spans": [ // all decoded spans (word indices [start, end)) |
| {"tag": "safety.pii.private_name", "family": "pii", "start": 3, "end": 4, "text": "Maria", "p": 0.97, "thr": 0.42, |
| "alt": {"tag": "...", "p": 0.01}, // runner-up type |
| "ver": 0.99, // span verifier probability |
| "src": "head"} // decoded by the v2 span head |
| ], |
| "entities": [...], "pii": [...], "profanity_toxicity": [...], // spans split by family |
| "redacted_text": "Hi, this is [NAME] from ...", // PII / profanity / toxicity spans masked (public_contact is not masked) |
| "utterances": [ |
| {"start": 0, "end": 8, "text": "...", |
| "intent": ["intent.request_information"], "act": ["discourse.act.question"], "safety": ["safety.pii.request"], |
| "role": [], "role_note": "speaker role needs diarization (no speaker id on words)", |
| "item": [], "state": {"content.mode": ["content.mode.call_in"]}, |
| "emotion": [{"tag": "affect.emotion.neutral", "p": 0.93, "thr": 0.6, "on": true}, ...], |
| "scores": {"intent": [{"tag": ..., "p": ..., "thr": ..., "on": ...}], "act": [...], ...}} // top-k with probabilities |
| ], |
| "events": [{"tag": "speaker.change", "at": 15, "word": "Yeah,", "p": 0.94, "thr": 0.3, "tier": "T0", "read": "ev_rec"}] |
| } |
| ``` |
|
|
| The label lists (`intent`, `act`, `safety`, ...) contain the tags at or above their threshold (`on`). `scores` keeps the top-k with |
| their probabilities. |
|
|
| Each streaming update (`push_words` and `endpoint`) returns: |
| * `n_words` and `changed_from` |
| * `spans`: the re-decoded window; replace your spans that end after its start |
| * `events` (new ones only) |
| * `utterances` (the ones that just ended) |
| * `running`: the live intent, act, safety and state of the open utterance |
|
|
| ## Latency tiers |
|
|
| | tier | lookahead | use | |
| |---|---|---| |
| | T0 | 0 words | live display and zero-lag redaction (default; thresholds fitted here) | |
| | T1 | up to 3 words | revisions a few words later | |
| | T2 | to utterance end (up to 8 words) | final utterance labels | |
|
|
| All three tiers are delayed read-outs of one causal pass, so they add no extra compute. Utterance-level labels (intent, act, safety, |
| role) are read at the endpoint. |
|
|
| ## Evaluation |
|
|
| Internal multi-domain dev and test sets cover contact centre, assistant, meetings, news, talk, sports and courts, using real and |
| simulated ASR (Deepgram and RNNT word streams). |
|
|
| | metric | old gold | cleaned gold | |
| |---|---|---| |
| | light eval, primary score (step 88000) | 0.693 | 0.764 | |
| | test, T0 macro AP | 0.719 | 0.793 | |
|
|
| **Caveat:** "cleaned gold" re-adjudicated the eval labels after a confident-learning label audit. The model selected the suspect |
| labels and Claude adjudicated them. The same audit also cleaned part of the training data, so cleaned-gold numbers are partly |
| favourable to this model family. Treat **old-gold numbers as the conservative estimate**. These are internal benchmarks, not public |
| leaderboards. |
|
|
| ## Files |
|
|
| | file | what | |
| |---|---| |
| | `model.safetensors` | weights (fp32, 45 MB) | |
| | `config.json` | model, ontology and read-out columns; training args with data paths removed | |
| | `tok/bpe8k/` | 8k byte-level BPE tokenizer (words are fed as `" " + word`) | |
| | `ontology/` | r8 ontology, text-tag table, `ext_cc_roles` extension, frozen tag-definition embeddings (bge-small) | |
| | `thresholds/step88000.json` | display thresholds: a precision-first T0 refit on the cleaned dev split, per tag, read-out and domain | |
| | `thresholds/events_step88000.json` | display tier, thresholds and hidden list for each event | |
| | `demo_settings.json` | the live demo's engine floors and settings (span threshold of at least .3; toxicity of at least .5 outside safety verticals) | |
| | `label_norm.json` | label-normalisation conventions behind the tags (training-time; pii1sn3) | |
| | `arctag/` | Python package: `ArcTagger`, `driver.py` (session to JSON), `_core/` (the demo's streaming engine and model code, vendored with only file paths changed) | |
| | `onnx/` | ONNX export for the C API, if present (published separately) | |
| | `MANIFEST.sha256` | checksums | |
|
|
| ## Limitations |
|
|
| * **English only.** Non-English input gives garbage-like output. |
| * Text only: no audio cues. The model cannot detect speaker identity, emotion from voice or vocal events. **Speaker roles need speaker |
| ids** (`spk`) on the words, for example from ASR diarization. |
| * Utterance-level safety and scam flags can fire on short or out-of-domain utterances; use `scores` (p) and your own policy for |
| anything consequential. Emotion is low-support and mostly predicts neutral. |
| * The `auto` domain is weaker than setting the domain. Some sports events fire loosely outside sports. |
| * Redaction here masks the decoded spans after the fact. The live demo's zero-lag redactor (carrier-phrase pre-arm, regex rules, hold |
| modes) is not included. |
| * Thresholds are fitted for precision first on internal dev data. Refit them for your traffic if you can. |
| * CPU inference only in this package, which is fp32. The ONNX export, if present, is the path for C and embedded use. |
|
|
| ## Licence |
|
|
| Weights and code, including all inference code: CC BY-NC 4.0 plus the Whissle terms in [LICENSE](LICENSE), non-commercial use |
| only. Commercial use needs a separate written licence from Whissle Inc.: ksingla@whissle.ai. Third-party components keep their |
| own licences. |
|
|
