| --- |
| library_name: premove-itn |
| language: |
| - en |
| base_model: |
| - microsoft/deberta-v3-large |
| tags: |
| - inverse-text-normalization |
| - text-normalization |
| - speech-processing |
| - voice-agents |
| - structured-prediction |
| - candidate-ranking |
| - safetensors |
| - custom-code |
| license: mit |
| --- |
| |
| # Premove ITN v0.1.0 |
|
|
| Premove ITN is an English inverse text normalization system for voice-agent |
| utterances. This release is the first frozen, inference-only artifact for its |
| structured-value contextual scorer. It ranks deterministic Rust candidates |
| using sentence context, then applies exact maximum-score decoding. |
|
|
| **Premove ITN is released as an open-weight contextual inverse text |
| normalization model. The inference code and model weights are licensed under |
| MIT.** |
|
|
| The artifact contains the complete trained parameters in `model.safetensors`, |
| the exact DeBERTa configuration, and the tokenizer files used by the selected |
| production checkpoint. Optimizer state, scheduler state, training counters, |
| training data, and evaluation rows are not included. |
|
|
| The artifact is loaded through the `premove-itn` package. It is not a generic |
| Transformers model; use `PremoveITN.from_pretrained()` so candidate generation, |
| scoring, and decoding stay aligned with the release implementation: |
|
|
| ```python |
| from premove_itn import PremoveITN |
| |
| itn = PremoveITN.from_pretrained( |
| "premove-itn/premove-itn", |
| revision="v0.1.0", |
| ) |
| print(itn.normalize("call me at four thirty")) # call me at 04:30 |
| ``` |
|
|
| Source code, deterministic Rust realization rules, the Python API, benchmark |
| code, and retained results are available in the |
| [`premove-ai/premove-itn`](https://github.com/premove-ai/premove-itn) |
| repository. |
|
|
| The model uses `microsoft/deberta-v3-large` at the exact revision recorded in |
| `provenance.json`. The artifact includes the tokenizer and base configuration; |
| the base model is not separately redistributed. Microsoft lists the base model |
| under the MIT license; retain that attribution when redistributing this |
| artifact. |
|
|
| This release is tagged `v0.1.0`. Do not move or overwrite that tag after |
| publication. The artifact digest and source checkpoint lineage are recorded in |
| `provenance.json`. |
|
|
| ## Intended use |
|
|
| Use this artifact for English voice-agent text normalization where spoken |
| numbers, dates, times, money, phone values, identifiers, URLs, and related |
| structured values require context-sensitive disambiguation. The model receives |
| only the utterance text at inference time. Candidate metadata is generated by |
| the package's deterministic Rust layer. |
|
|
| ## Supported span kinds |
|
|
| The supported span kinds are listed in `config.json`: `DIGIT_SEQUENCE`, |
| `CARDINAL`, `TIME`, `DATE`, `MONEY`, `DECIMAL`, `PHONE`, `ELECTRONIC`, |
| `MEASUREMENT`, `ORDINAL`, `PUNCTUATION`, `WHITELIST`, and `WORD`. |
|
|
| ## Runtime |
|
|
| The contextual API requires Python 3.11 or newer, PyTorch, Transformers, |
| SentencePiece, safetensors, and huggingface_hub. `device="auto"` selects CUDA, |
| then Apple MPS, then CPU. Pass `device="cpu"`, `device="mps"`, or |
| `device="cuda"` to select a device explicitly. The retained latency result was |
| measured with batch size one on Apple MPS. Other runtimes and devices can have |
| different latency and memory use. |
| |
| ## First Evaluation |
| |
| The retained First Evaluation used 1,500 frozen VoiceAgent ITN rows and three |
| backends. The metrics below are semantic entity metrics unless marked strict. |
| Formatting-only differences can reduce strict exact match while preserving the |
| same semantic value. |
| |
| | Backend | Entity micro | Entity macro | Voice rows | Mean latency | |
| |---|---:|---:|---:|---:| |
| | Premove ITN | 89.70% | 90.71% | 398/400 (99.50%) | 56.49 ms | |
| | Thutmose | 59.39% | 57.61% | 268/400 (67.00%) | 15.98 ms | |
| | text-processing-rs | 55.79% | 55.05% | 273/400 (68.25%) | 0.14 ms | |
| |
| Premove leads the measured semantic accuracy overall and on the 400 |
| voice-agent-domain rows. It does not lead latency. This is a balanced synthetic |
| stress benchmark, not an estimate of production traffic accuracy, and the run |
| was not blind. |
| |
| ## Limitations |
| |
| - This custom scorer is not directly loadable with `AutoModel.from_pretrained`. |
| Use `PremoveITN.from_pretrained()` and the matching `premove-itn` package |
| version. |
| - The artifact is English-only and depends on the package's candidate graph and |
| decoder for end-to-end normalization. |
| - Benchmark results are not a guarantee for unseen domains or formatting |
| policies. Human gold adjudication and broader production-distribution testing |
| remain separate requirements. |
|
|
| ## Release identity |
|
|
| - Artifact version: `v0.1.0` |
| - Architecture: `premove-candidate-scorer-v1` |
| - Package version: `0.1.0` |
| - Hub repository: `premove-itn/premove-itn` |
| - Hub revision: `v0.1.0` |
| - Hub commit: `80bda5e2e1fe9542aa628597090242df57c1a157` |
| - Base model: `microsoft/deberta-v3-large` |
| - Base model revision: `64a8c8eab3e352a784c658aef62be1662607476f` |
| - Source checkpoint SHA-256: `9021fa11a028faefb31ef67878170cbe29ed25e68a9a78999f37b120c2ad00d5` |
| - Inference artifact SHA-256: `119c0f19767b61446e04da1f8f01a001edf97a47a66965e7146db2483b4937a1` |
|
|
| ## Licensing |
|
|
| Premove ITN is MIT licensed. The pinned base model is [`microsoft/deberta-v3-large`](https://huggingface.co/microsoft/deberta-v3-large), whose model card lists an MIT license. The architecture derives from [DeBERTaV3](https://arxiv.org/abs/2111.09543). Review and retain both notices when redistributing this artifact. |
|
|
