README.md

---
library_name: premove-itn
language:
  - en
base_model:
  - microsoft/deberta-v3-large
tags:
  - inverse-text-normalization
  - text-normalization
  - speech-processing
  - voice-agents
  - structured-prediction
  - candidate-ranking
  - safetensors
  - custom-code
license: mit
---

# Premove ITN v0.1.0

Premove ITN is an English inverse text normalization system for voice-agent
utterances. This release is the first frozen, inference-only artifact for its
structured-value contextual scorer. It ranks deterministic Rust candidates
using sentence context, then applies exact maximum-score decoding.

**Premove ITN is released as an open-weight contextual inverse text
normalization model. The inference code and model weights are licensed under
MIT.**

The artifact contains the complete trained parameters in `model.safetensors`,
the exact DeBERTa configuration, and the tokenizer files used by the selected
production checkpoint. Optimizer state, scheduler state, training counters,
training data, and evaluation rows are not included.

The artifact is loaded through the `premove-itn` package. It is not a generic
Transformers model; use `PremoveITN.from_pretrained()` so candidate generation,
scoring, and decoding stay aligned with the release implementation:

```python
from premove_itn import PremoveITN

itn = PremoveITN.from_pretrained(
    "premove-itn/premove-itn",
    revision="v0.1.0",
)
print(itn.normalize("call me at four thirty"))  # call me at 04:30
```

Source code, deterministic Rust realization rules, the Python API, benchmark
code, and retained results are available in the
[`premove-ai/premove-itn`](https://github.com/premove-ai/premove-itn)
repository.

The model uses `microsoft/deberta-v3-large` at the exact revision recorded in
`provenance.json`. The artifact includes the tokenizer and base configuration;
the base model is not separately redistributed. Microsoft lists the base model
under the MIT license; retain that attribution when redistributing this
artifact.

This release is tagged `v0.1.0`. Do not move or overwrite that tag after
publication. The artifact digest and source checkpoint lineage are recorded in
`provenance.json`.

## Intended use

Use this artifact for English voice-agent text normalization where spoken
numbers, dates, times, money, phone values, identifiers, URLs, and related
structured values require context-sensitive disambiguation. The model receives
only the utterance text at inference time. Candidate metadata is generated by
the package's deterministic Rust layer.

## Supported span kinds

The supported span kinds are listed in `config.json`: `DIGIT_SEQUENCE`,
`CARDINAL`, `TIME`, `DATE`, `MONEY`, `DECIMAL`, `PHONE`, `ELECTRONIC`,
`MEASUREMENT`, `ORDINAL`, `PUNCTUATION`, `WHITELIST`, and `WORD`.

## Runtime

The contextual API requires Python 3.11 or newer, PyTorch, Transformers,
SentencePiece, safetensors, and huggingface_hub. `device="auto"` selects CUDA,
then Apple MPS, then CPU. Pass `device="cpu"`, `device="mps"`, or
`device="cuda"` to select a device explicitly. The retained latency result was
measured with batch size one on Apple MPS. Other runtimes and devices can have
different latency and memory use.

## First Evaluation

The retained First Evaluation used 1,500 frozen VoiceAgent ITN rows and three
backends. The metrics below are semantic entity metrics unless marked strict.
Formatting-only differences can reduce strict exact match while preserving the
same semantic value.

| Backend | Entity micro | Entity macro | Voice rows | Mean latency |
|---|---:|---:|---:|---:|
| Premove ITN | 89.70% | 90.71% | 398/400 (99.50%) | 56.49 ms |
| Thutmose | 59.39% | 57.61% | 268/400 (67.00%) | 15.98 ms |
| text-processing-rs | 55.79% | 55.05% | 273/400 (68.25%) | 0.14 ms |

Premove leads the measured semantic accuracy overall and on the 400
voice-agent-domain rows. It does not lead latency. This is a balanced synthetic
stress benchmark, not an estimate of production traffic accuracy, and the run
was not blind.

## Limitations

- This custom scorer is not directly loadable with `AutoModel.from_pretrained`.
  Use `PremoveITN.from_pretrained()` and the matching `premove-itn` package
  version.
- The artifact is English-only and depends on the package's candidate graph and
  decoder for end-to-end normalization.
- Benchmark results are not a guarantee for unseen domains or formatting
  policies. Human gold adjudication and broader production-distribution testing
  remain separate requirements.

## Release identity

- Artifact version: `v0.1.0`
- Architecture: `premove-candidate-scorer-v1`
- Package version: `0.1.0`
- Hub repository: `premove-itn/premove-itn`
- Hub revision: `v0.1.0`
- Hub commit: `80bda5e2e1fe9542aa628597090242df57c1a157`
- Base model: `microsoft/deberta-v3-large`
- Base model revision: `64a8c8eab3e352a784c658aef62be1662607476f`
- Source checkpoint SHA-256: `9021fa11a028faefb31ef67878170cbe29ed25e68a9a78999f37b120c2ad00d5`
- Inference artifact SHA-256: `119c0f19767b61446e04da1f8f01a001edf97a47a66965e7146db2483b4937a1`

## Licensing

Premove ITN is MIT licensed. The pinned base model is [`microsoft/deberta-v3-large`](https://huggingface.co/microsoft/deberta-v3-large), whose model card lists an MIT license. The architecture derives from [DeBERTaV3](https://arxiv.org/abs/2111.09543). Review and retain both notices when redistributing this artifact.