Hugging Face's logo

premove-ai
/
premove-itn

Safetensors
English
premove-itn
inverse-text-normalization
speech-processing
voice-agents
Model card Files
xet
 Community
4
premove-itn / README.md
aryaman20's picture
aryaman20
Document open-weight release and public loading API
4ce5f73 verified
preview code
|
Raw History Blame
5.39 kB
metadata 
library_name: premove-itn
language:
  - en
base_model:
  - microsoft/deberta-v3-large
tags:
  - inverse-text-normalization
  - text-normalization
  - speech-processing
  - voice-agents
  - structured-prediction
  - candidate-ranking
  - safetensors
  - custom-code
license: mit

Premove ITN v0.1.0

Premove ITN is an English inverse text normalization system for voice-agent utterances. This release is the first frozen, inference-only artifact for its structured-value contextual scorer. It ranks deterministic Rust candidates using sentence context, then applies exact maximum-score decoding.

Premove ITN is released as an open-weight contextual inverse text normalization model. The inference code and model weights are licensed under MIT.

The artifact contains the complete trained parameters in model.safetensors, the exact DeBERTa configuration, and the tokenizer files used by the selected production checkpoint. Optimizer state, scheduler state, training counters, training data, and evaluation rows are not included.

The artifact is loaded through the premove-itn package. It is not a generic Transformers model; use PremoveITN.from_pretrained() so candidate generation, scoring, and decoding stay aligned with the release implementation:

from premove_itn import PremoveITN

itn = PremoveITN.from_pretrained(
    "premove-itn/premove-itn",
    revision="v0.1.0",
)
print(itn.normalize("call me at four thirty"))  # call me at 04:30

Source code, deterministic Rust realization rules, the Python API, benchmark code, and retained results are available in the premove-ai/premove-itn repository.

The model uses microsoft/deberta-v3-large at the exact revision recorded in provenance.json. The artifact includes the tokenizer and base configuration; the base model is not separately redistributed. Microsoft lists the base model under the MIT license; retain that attribution when redistributing this artifact.

This release is tagged v0.1.0. Do not move or overwrite that tag after publication. The artifact digest and source checkpoint lineage are recorded in provenance.json.

Intended use

Use this artifact for English voice-agent text normalization where spoken numbers, dates, times, money, phone values, identifiers, URLs, and related structured values require context-sensitive disambiguation. The model receives only the utterance text at inference time. Candidate metadata is generated by the package's deterministic Rust layer.

Supported span kinds

The supported span kinds are listed in config.json: DIGIT_SEQUENCE, CARDINAL, TIME, DATE, MONEY, DECIMAL, PHONE, ELECTRONIC, MEASUREMENT, ORDINAL, PUNCTUATION, WHITELIST, and WORD.

Runtime

The contextual API requires Python 3.11 or newer, PyTorch, Transformers, SentencePiece, safetensors, and huggingface_hub. device="auto" selects CUDA, then Apple MPS, then CPU. Pass device="cpu", device="mps", or device="cuda" to select a device explicitly. The retained latency result was measured with batch size one on Apple MPS. Other runtimes and devices can have different latency and memory use.

First Evaluation

The retained First Evaluation used 1,500 frozen VoiceAgent ITN rows and three backends. The metrics below are semantic entity metrics unless marked strict. Formatting-only differences can reduce strict exact match while preserving the same semantic value.

Backend Entity micro Entity macro Voice rows Mean latency
Premove ITN 89.70% 90.71% 398/400 (99.50%) 56.49 ms
Thutmose 59.39% 57.61% 268/400 (67.00%) 15.98 ms
text-processing-rs 55.79% 55.05% 273/400 (68.25%) 0.14 ms

Premove leads the measured semantic accuracy overall and on the 400 voice-agent-domain rows. It does not lead latency. This is a balanced synthetic stress benchmark, not an estimate of production traffic accuracy, and the run was not blind.

Limitations

  • This custom scorer is not directly loadable with AutoModel.from_pretrained. Use PremoveITN.from_pretrained() and the matching premove-itn package version.
  • The artifact is English-only and depends on the package's candidate graph and decoder for end-to-end normalization.
  • Benchmark results are not a guarantee for unseen domains or formatting policies. Human gold adjudication and broader production-distribution testing remain separate requirements.

Release identity

  • Artifact version: v0.1.0
  • Architecture: premove-candidate-scorer-v1
  • Package version: 0.1.0
  • Hub repository: premove-itn/premove-itn
  • Hub revision: v0.1.0
  • Hub commit: 80bda5e2e1fe9542aa628597090242df57c1a157
  • Base model: microsoft/deberta-v3-large
  • Base model revision: 64a8c8eab3e352a784c658aef62be1662607476f
  • Source checkpoint SHA-256: 9021fa11a028faefb31ef67878170cbe29ed25e68a9a78999f37b120c2ad00d5
  • Inference artifact SHA-256: 119c0f19767b61446e04da1f8f01a001edf97a47a66965e7146db2483b4937a1

Licensing

Premove ITN is MIT licensed. The pinned base model is microsoft/deberta-v3-large, whose model card lists an MIT license. The architecture derives from DeBERTaV3. Review and retain both notices when redistributing this artifact.