Blame for README.md

---
language:
- en
license: cc-by-nc-4.0
library_name: pytorch
pipeline_tag: token-classification
tags:
- streaming
- asr
- ner
- pii
- redaction
- intent
- dialogue-act
- mamba
---

# Arc Tagger (10M), v0.1.0

Arc Tagger is a **text-only, streaming tagger** with about 9.8 M parameters. It reads the word stream of **any ASR** (Deepgram, Nemotron,
Whisper, our own ASR, or plain text). As each word arrives, it emits entity and number spans, PII spans for redaction, profanity and
toxicity, discourse acts, intents, safety flags, speaker roles, events (speaker change, turn yield, sports, legal and meeting events),
content state and emotion (experimental).

* Checkpoint: `AT10Mv2` step 88000. It scored best on all gold sets of its run.
* Architecture: a word channel (8k byte-level BPE, 2 causal local-attention layers) feeds a 6-layer Mamba-3 trunk. Global KV-memory
  attention layers, domain FiLM and adapters condition the trunk. The read-outs are definition-keyed tag heads plus a v2 span head (BIO
  and type per family), a span verifier and pooled utterance, speaker and item heads. The model is causal and runs one recurrent step
  per word, so a revision re-runs only the revised words.
* Runs on CPU. Measured on one core (busy shared box, 1 thread): **engine about 16 ms per word p50, 25 ms p95**. A full
  `push_words()` call, including span decoding and read-outs, takes about 24 ms p50 and 45 ms p95.

## Tag families

276 tags across 10 read-outs. Every tag name comes from the Whissle ontology r8 plus the extension `ext_cc_roles1`.

| family | tags (examples) | read-out |
|---|---|---|
| Entities | `entity.mention.{person,organization,team,place,event_name,legal_ref,term,...}` (10) | word spans |
| Numbers | `entity.number.{date,time_of_day,duration,count,money,percent,measure,score,clock,ordinal,id_pii,...}` (16), `entity.numeral` | word spans |
| PII / redaction | `safety.pii.{private_name,phone_number,email,street_address,card_account_number,gov_id,date_of_birth,url_handle,public_contact}` spans; `safety.pii.{request,leak}` per utterance | spans and utterance |
| Profanity / toxicity | `safety.profanity.{mild,strong}`, `safety.toxicity.*` (6) | spans and utterance |
| Safety | `safety.scam.*` (6), `safety.crisis.self_harm`, `safety.threat.escalation`, `safety.child.*`, `safety.solicitation.dating_romantic` | utterance |
| Discourse act | `discourse.act.*` (statement, question, command_request, ...), `discourse.answer`, `discourse.backchannel`, `discourse.assistant.*` | utterance |
| Intent | 26 cross-domain `intent.*` (request_information, schedule_arrange, complain_criticize, ...), 60 assistant `discourse.intent.*` (alarm_set, weather_query, ...) | utterance |
| Speaker role | `speaker.role.*` (19, including contact-centre agent and customer) | per speaker (needs `spk` ids on the words) |
| Events | `speaker.change`, `discourse.turn_yield`, `discourse.interruption`, `event.sports.*` (18), `event.legal.*`, `event.meeting.{decision,action_item}` | per word |
| Content state | `content.mode`, `content.subject`, `content.game_state`, `content.legal_phase`, `content.meeting_phase` | per word (frame) |
| Emotion (experimental) | `affect.emotion.*` (5), `affect.arousal.*` (3) | utterance; low support, mostly "neutral" |

`ontology/` holds the full definitions. `config.json["columns"]` lists every `tag@readout` column.

## Quickstart

```bash
pip install huggingface_hub
huggingface-cli download WhissleAI/arc-tagger --local-dir arc-tagger     # private repo: huggingface-cli login first
cd arc-tagger
pip install -r requirements.txt     # CPU torch is fine: pip install torch --index-url https://download.pytorch.org/whl/cpu
python example.py
```

```python
import sys; sys.path.insert(0, "arc-tagger")          # the package ships in the repo (arctag/)
from arctag import ArcTagger

tg = ArcTagger.from_pretrained("arc-tagger")           # a local dir, or a repo id: ArcTagger.from_pretrained("WhissleAI/arc-tagger")
out = tg.tag("Hi, this is Maria from Northwind Bank. Can I get your phone number?", domain="talk")
print(out["redacted_text"]); print(out["spans"]); print(out["utterances"])

# streaming (words as the ASR emits them; interim hypotheses can be revised)
st = tg.stream(domain="assistant")
st.push_words(["set", "an", "alarm", "for"], final=False)                       # interim
up = st.push_words(["set", "an", "alarm", "for", "seven", "thirty."], revise_from=0, final=True)
up = st.endpoint()                         # end of utterance -> utterance read-outs in up["utterances"]
res = st.finalize()                        # full result (same schema as tag())
```

### Input

`tag(x, domain="auto", tier="T0")`
* `x` can be a `str`, a list of words, or a list of word dicts `{"w": str, "t0": s, "t1": s, "conf": float, "spk": str|int, "final": bool}`.
  Every field except `w` is optional.
* Utterances end after words ending in `. ? !`. In streaming, call `endpoint()` at the ASR's end of utterance or end of turn.

`domain` is one of `auto | talk | meeting | court | assistant | sports | news`.
* It conditions the model through domain FiLM and adapters, and picks the per-domain thresholds.
* `talk` covers contact centre, podcasts and chat. Inside `sports` and `court`, the specific league or court type is detected from
  lexicon cues.
* `auto` runs the "unknown" domain, which is weaker. **Set the domain when you know it.**

`tier`
* `T0` (default; 0 words of lookahead; the thresholds were fitted on it)
* `T1` (at most 3 words of delay)
* `T2` (utterance end)

Streaming: `push_words(words, final=True, revise_from=None, endpoint=False)`
* `revise_from=k` replaces `words[k:]`. Use it for interim hypotheses such as Deepgram's `is_final=false` or a growing RNNT hypothesis.

### Output (JSON)

```jsonc
{
 "meta": {"model": "arc-tagger", "version": "0.1.0", "step": 88000, "domain": "talk", "fine_domain": "talk", "tier": "T0", "thresholds": "..."},
 "words": ["Hi,", "this", "is", "Maria", ...],
 "spans": [                                     // all decoded spans (word indices [start, end))
   {"tag": "safety.pii.private_name", "family": "pii", "start": 3, "end": 4, "text": "Maria", "p": 0.97, "thr": 0.42,
    "alt": {"tag": "...", "p": 0.01},           // runner-up type
    "ver": 0.99,                                // span verifier probability
    "src": "head"}                              // decoded by the v2 span head
 ],
 "entities": [...], "pii": [...], "profanity_toxicity": [...],   // spans split by family
 "redacted_text": "Hi, this is [NAME] from ...",                  // PII / profanity / toxicity spans masked (public_contact is not masked)
 "utterances": [
   {"start": 0, "end": 8, "text": "...",
    "intent": ["intent.request_information"], "act": ["discourse.act.question"], "safety": ["safety.pii.request"],
    "role": [], "role_note": "speaker role needs diarization (no speaker id on words)",
    "item": [], "state": {"content.mode": ["content.mode.call_in"]},
    "emotion": [{"tag": "affect.emotion.neutral", "p": 0.93, "thr": 0.6, "on": true}, ...],
    "scores": {"intent": [{"tag": ..., "p": ..., "thr": ..., "on": ...}], "act": [...], ...}}   // top-k with probabilities
 ],
 "events": [{"tag": "speaker.change", "at": 15, "word": "Yeah,", "p": 0.94, "thr": 0.3, "tier": "T0", "read": "ev_rec"}]
}
```

The label lists (`intent`, `act`, `safety`, ...) contain the tags at or above their threshold (`on`). `scores` keeps the top-k with
their probabilities.

Each streaming update (`push_words` and `endpoint`) returns:
* `n_words` and `changed_from`
* `spans`: the re-decoded window; replace your spans that end after its start
* `events` (new ones only)
* `utterances` (the ones that just ended)
* `running`: the live intent, act, safety and state of the open utterance

## Latency tiers

| tier | lookahead | use |
|---|---|---|
| T0 | 0 words | live display and zero-lag redaction (default; thresholds fitted here) |
| T1 | up to 3 words | revisions a few words later |
| T2 | to utterance end (up to 8 words) | final utterance labels |

All three tiers are delayed read-outs of one causal pass, so they add no extra compute. Utterance-level labels (intent, act, safety,
role) are read at the endpoint.

## Evaluation

Internal multi-domain dev and test sets cover contact centre, assistant, meetings, news, talk, sports and courts, using real and
simulated ASR (Deepgram and RNNT word streams).

| metric | old gold | cleaned gold |
|---|---|---|
| light eval, primary score (step 88000) | 0.693 | 0.764 |
| test, T0 macro AP | 0.719 | 0.793 |

**Caveat:** "cleaned gold" re-adjudicated the eval labels after a confident-learning label audit. The model selected the suspect
labels and Claude adjudicated them. The same audit also cleaned part of the training data, so cleaned-gold numbers are partly
favourable to this model family. Treat **old-gold numbers as the conservative estimate**. These are internal benchmarks, not public
leaderboards.

## Files

| file | what |
|---|---|
| `model.safetensors` | weights (fp32, 45 MB) |
| `config.json` | model, ontology and read-out columns; training args with data paths removed |
| `tok/bpe8k/` | 8k byte-level BPE tokenizer (words are fed as `" " + word`) |
| `ontology/` | r8 ontology, text-tag table, `ext_cc_roles` extension, frozen tag-definition embeddings (bge-small) |
| `thresholds/step88000.json` | display thresholds: a precision-first T0 refit on the cleaned dev split, per tag, read-out and domain |
| `thresholds/events_step88000.json` | display tier, thresholds and hidden list for each event |
| `demo_settings.json` | the live demo's engine floors and settings (span threshold of at least .3; toxicity of at least .5 outside safety verticals) |
| `label_norm.json` | label-normalisation conventions behind the tags (training-time; pii1sn3) |
| `arctag/` | Python package: `ArcTagger`, `driver.py` (session to JSON), `_core/` (the demo's streaming engine and model code, vendored with only file paths changed) |
| `onnx/` | ONNX export for the C API, if present (published separately) |
| `MANIFEST.sha256` | checksums |

## Limitations

* **English only.** Non-English input gives garbage-like output.
* Text only: no audio cues. The model cannot detect speaker identity, emotion from voice or vocal events. **Speaker roles need speaker
  ids** (`spk`) on the words, for example from ASR diarization.
* Utterance-level safety and scam flags can fire on short or out-of-domain utterances; use `scores` (p) and your own policy for
  anything consequential. Emotion is low-support and mostly predicts neutral.
* The `auto` domain is weaker than setting the domain. Some sports events fire loosely outside sports.
* Redaction here masks the decoded spans after the fact. The live demo's zero-lag redactor (carrier-phrase pre-arm, regex rules, hold
  modes) is not included.
* Thresholds are fitted for precision first on internal dev data. Refit them for your traffic if you can.
* CPU inference only in this package, which is fp32. The ONNX export, if present, is the path for C and embedded use.

## Licence

Weights and code, including all inference code: CC BY-NC 4.0 plus the Whissle terms in [LICENSE](LICENSE), non-commercial use
only. Commercial use needs a separate written licence from Whissle Inc.: ksingla@whissle.ai. Third-party components keep their
own licences.