Publish Premove ITN v0.2.0 model

README.md

@@ -1,178 +1,68 @@
 ---
 library_name: premove-itn
-language:
-  - en
-base_model:
-  - microsoft/deberta-v3-large
-tags:
-  - inverse-text-normalization
-  - text-normalization
-  - speech-processing
-  - voice-agents
-  - structured-prediction
-  - candidate-ranking
-  - safetensors
-  - custom-code
+language: [en]
+base_model: [microsoft/deberta-v3-large]
+tags: [inverse-text-normalization, speech-processing, voice-agents]
 license: mit
 ---
 
-# Premove ITN v0.1.0
+# Premove ITN v0.2.0
 
-Premove ITN is an open-weight contextual inverse text normalization system for
-English voice-agent transcripts. It is the contextual scoring model used by the
-`premove-itn` Python package. It turns spoken-form ASR text into structured
-written text:
+This release updates the contextual candidate scorer used by `premove-itn`.
+The deterministic Rust candidate generators and exact decoder are unchanged.
 
-```text
-call me at four thirty  →  call me at 04:30
-the total is twenty dollars  →  the total is $20
-```
-
-Deterministic Rust realizers propose valid written forms. A
-DeBERTa-v3-large contextual scorer uses the complete sentence to score those
-candidates, and an exact decoder selects compatible, non-overlapping edits.
-
-This repository contains the frozen, inference-only v0.1.0 model artifact. It
-does not contain optimizer state, scheduler state, training counters, training
-data, or evaluation rows. Source code and retained evaluation evidence are in
-[`premove-ai/premove-itn`](https://github.com/premove-ai/premove-itn).
-
-## Loading the model
-
-This is a custom candidate-scoring architecture. Do not load it with
-`AutoModel.from_pretrained()`.
+Load it through the matching Python package:
 
 ```python
 from premove_itn import PremoveITN
 
-itn = PremoveITN.from_pretrained()
-print(itn.normalize("call me at four thirty"))
-# call me at 04:30
+itn = PremoveITN.from_pretrained(revision="v0.2.0")
+print(itn.normalize("meet me at two thirty"))
+# meet me at 02:30
 ```
 
-Install the latest package with `pip install premove-itn`, or pin the current
-release with `pip install premove-itn==0.1.0`. Create one
-`PremoveITN` instance and reuse it; model initialization is expensive compared
-with warm normalization.
-
-`device="auto"` selects CUDA when available, then Apple MPS, then CPU. The
-Release wheels are validated on macOS 14+ arm64 and `manylinux_2_28` x86_64
-for Python 3.11–3.13. Real frozen-model inference is validated on Apple
-Silicon MPS and Linux CPU. Other environments require release validation. The source repository retains the current
-[platform support matrix](https://github.com/premove-ai/premove-itn/blob/main/docs/platform-support.md).
-
-## Architecture
-
-```text
-Spoken ASR text
-      ↓
-deterministic Rust candidates
-      ↓
-DeBERTa-v3-large contextual scores
-      ↓
-exact maximum-score decoder
-      ↓
-written transcript
-```
-
-The scorer has 435,594,145 parameters. The artifact contains the complete
-trained state in `model.safetensors`, the pinned DeBERTa configuration, and the
-tokenizer files required by the release.
-
-Supported candidate kinds are `DIGIT_SEQUENCE`, `CARDINAL`, `TIME`, `DATE`,
-`MONEY`, `DECIMAL`, `PHONE`, `ELECTRONIC`, `MEASUREMENT`, `ORDINAL`,
-`PUNCTUATION`, `WHITELIST`, and `WORD`.
-
-## First Evaluation
+## Training
 
-The retained First Evaluation used a frozen, balanced synthetic stress suite.
-Semantic entity accuracy is the primary structured-value metric. Strict exact
-match separately measures the complete canonical output.
+The v0.1.0 scorer was adapted for three epochs on 7,440 controlled,
+single-collision TIME-versus-identifier records. No replay or dual-collision
+examples were used. Epoch 3 was selected using a separate 200-row development
+set before any frozen test was inspected.
 
-### Dedicated voice-agent rows
+## Results
 
-| Backend | Correct entities | Semantic accuracy | Mean latency |
-| --- | ---: | ---: | ---: |
-| **Premove ITN** | **398/400** | **99.50%** | 57.41 ms |
-| Thutmose | 268/400 | 67.00% | 16.04 ms |
-| text-processing-rs | 273/400 | 68.25% | 0.15 ms |
+| Metric | v0.1.0 | v0.2.0 |
+| --- | ---: | ---: |
+| Context record | 51.2% | **86.2%** |
+| Counterfactual pair | 6.4% | **72.4%** |
+| Identifier | 29.6% | **98.0%** |
+| Time | 72.8% | **74.4%** |
+| Dual span | 45.0% | **96.0%** |
+| Dual sentence exact | 16.0% | **92.0%** |
+| Broad strict exact | 40.5% | **43.7%** |
 
-### Overall 1,500-row benchmark
-
-| Backend | Semantic accuracy | Strict exact | Mean latency |
-| --- | ---: | ---: | ---: |
-| **Premove ITN** | **89.70%** | **40.53%** | 56.49 ms |
-| Thutmose | 59.39% | 22.13% | 15.98 ms |
-| text-processing-rs | 55.79% | 16.53% | 0.14 ms |
-
-Premove ITN led measured semantic accuracy overall and on the 400 dedicated
-voice-agent rows. It did not lead latency.
-
-Latency used sequential batch-one requests on an Apple M4 MacBook Air with
-MPS, an optimized Rust extension, and eight Rayon workers. Models were loaded
-and warmed before request latency was measured. Download and initialization
-are excluded. The Premove ITN timing used the release Rust extension. The backend
-evaluation was blind: each backend received only transcript text.
-Independent human gold adjudication is a separate task and remains pending.
-
-See the
-[`First Evaluation report`](https://github.com/premove-ai/premove-itn/blob/main/eval/voice_agent_itn/results/first-evaluation/REPORT.md)
-and
-[`detailed tables`](https://github.com/premove-ai/premove-itn/blob/main/eval/voice_agent_itn/results/first-evaluation/DETAILS.md).
-
-## Intended use
-
-Use Premove ITN for English voice-agent transcripts in which numbers, dates,
-times, money, phone values, identifiers, URLs, and related structured values
-need sentence-level disambiguation. The runtime receives only transcript text.
-Candidate metadata is generated internally.
-
-## Model lifecycle
-
-- The first use downloads about 1.6 GB; duration depends on the network.
-- Cached initialization takes several seconds on the tested system.
-- Warm normalization averaged 56.49 ms with the retained release Rust build in
-  the MPS benchmark.
-- Services and transcript streams should keep one normalizer resident.
+The contextual and dual benchmarks are synthetic controlled evaluations. They
+do not estimate production voice-agent accuracy. The broad benchmark gained 86
+new exact rows and lost 38 previously exact rows. Localized losses were most
+visible in ORDINAL, MONEY, URL, and DIGIT_SEQUENCE.
 
 ## Limitations
 
-- English only.
-- A 435.6M-parameter model with an approximately 1.6 GB download.
-- Multi-second initialization.
-- Not compatible with generic `AutoModel.from_pretrained()` loading.
-- The benchmark is synthetic and does not measure live production traffic.
-- Weaker measured categories include URL, MONEY, CARDINAL, TIME,
-  REFERENCE_ID, and VERSION.
-- Independent human gold adjudication and broader contamination checks are
-  outside the v0.1.0 release gate. The backend evaluation itself was blind.
-- CUDA, Windows, macOS Intel, Linux ARM64, and other accelerators are not
-  validated v0.1.0 support claims.
+- TIME recall on the controlled single-collision test is 74.4%, substantially
+  below the 98.0% identifier result.
+- The model is English-only and requires the `premove-itn` candidate graph and
+  decoder. It is not a generic Transformers model.
+- The approximately 435.6M-parameter scorer has a large download and
+  multi-second initialization cost.
 
-## Release identity and provenance
+## Release identity
 
-- Artifact version: `v0.1.0`
-- Architecture: `premove-candidate-scorer-v1`
-- Required package version: `0.1.0`
+- Artifact version: `v0.2.0`
+- Required package version: `0.2.0`
 - Hub repository: `premove-ai/premove-itn`
-- Immutable model commit: `80bda5e2e1fe9542aa628597090242df57c1a157`
 - Base model: `microsoft/deberta-v3-large`
-- Base model revision: `64a8c8eab3e352a784c658aef62be1662607476f`
-- Model SHA-256: `119c0f19767b61446e04da1f8f01a001edf97a47a66965e7146db2483b4937a1`
-
-The package pins the immutable model commit and verifies its release metadata,
-base-model identity, and model digest before inference. Full training
-composition and checkpoint selection evidence are in the
-[`production model record`](https://github.com/premove-ai/premove-itn/blob/main/docs/model-provenance.md).
+- Base revision: `64a8c8eab3e352a784c658aef62be1662607476f`
+- Source checkpoint SHA-256: `10a338d57b6d619f52259a58aca210ecb78f47de44baf0c25477a8f5b06b5d39`
+- Model SHA-256: `0b6f36aa32311d0c495e1c5307d5e34463e52b4dc283ab9030bc2f54e1bf1152`
 
-## License and attribution
-
-Premove ITN source code and model weights are MIT licensed. The scorer uses
-[`microsoft/deberta-v3-large`](https://huggingface.co/microsoft/deberta-v3-large)
-at the revision above. Its architecture derives from the
-[`DeBERTaV3` paper](https://arxiv.org/abs/2111.09543).
-
-The Rust realization layer uses
-[`text-processing-rs`](https://github.com/FluidInference/text-processing-rs),
-which is Apache-2.0 licensed. Required notices are retained in the source
-repository.
+Source and evaluation evidence are available from
+[`premove-ai/premove-itn`](https://github.com/premove-ai/premove-itn).

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:119c0f19767b61446e04da1f8f01a001edf97a47a66965e7146db2483b4937a1
+oid sha256:0b6f36aa32311d0c495e1c5307d5e34463e52b4dc283ab9030bc2f54e1bf1152
 size 1742425916

provenance.json

@@ -1,23 +1,36 @@
 {
   "architecture": "premove-candidate-scorer-v1",
-  "artifact_sha256": "119c0f19767b61446e04da1f8f01a001edf97a47a66965e7146db2483b4937a1",
+  "artifact_sha256": "0b6f36aa32311d0c495e1c5307d5e34463e52b4dc283ab9030bc2f54e1bf1152",
   "artifact_sha256_definition": "SHA-256 of model.safetensors",
-  "artifact_version": "v0.1.0",
+  "artifact_version": "v0.2.0",
   "base_model": "microsoft/deberta-v3-large",
   "base_model_revision": "64a8c8eab3e352a784c658aef62be1662607476f",
-  "benchmark_checkpoint_sha256": "9021fa11a028faefb31ef67878170cbe29ed25e68a9a78999f37b120c2ad00d5",
-  "hub_repository": "premove-itn/premove-itn",
-  "hub_revision": "v0.1.0",
-  "package_version": "0.1.0",
-  "production_manifest": "data/models/structured_value_selected_20k/manifest.json",
+  "benchmark_checkpoint_sha256": "10a338d57b6d619f52259a58aca210ecb78f47de44baf0c25477a8f5b06b5d39",
+  "benchmark_results": {
+    "broad_strict_exact_accuracy": 0.43733333333333335,
+    "context_pair_accuracy": 0.724,
+    "context_record_accuracy": 0.862,
+    "dual_sentence_exact_accuracy": 0.92,
+    "dual_span_accuracy": 0.96,
+    "identifier_accuracy": 0.98,
+    "time_accuracy": 0.744
+  },
+  "hub_repository": "premove-ai/premove-itn",
+  "hub_revision": "v0.2.0",
+  "package_version": "0.2.0",
+  "production_manifest": "data/models/time_identifier_targeted_run_1/manifest.json",
   "redistribution": {
     "base_model_license": "MIT (as listed by microsoft/deberta-v3-large)",
     "base_model_weights_included": false,
     "project_license": "MIT",
     "tokenizer_files_included": true
   },
-  "source_checkpoint": "data/models/structured_value_selected_20k/checkpoint.pt",
-  "source_checkpoint_sha256": "9021fa11a028faefb31ef67878170cbe29ed25e68a9a78999f37b120c2ad00d5",
+  "release_commit": null,
+  "release_status": "candidate",
+  "source_checkpoint": "dist/time_identifier_targeted_run_1/epoch_3.pt",
+  "source_checkpoint_sha256": "10a338d57b6d619f52259a58aca210ecb78f47de44baf0c25477a8f5b06b5d39",
+  "source_commit": "d8240652fe92906f4fdf5aea280acf78e4e24221",
+  "source_worktree_dirty": false,
   "supported_span_kinds": [
     "DIGIT_SEQUENCE",
     "CARDINAL",
@@ -34,8 +47,22 @@
     "WORD"
   ],
   "tokenizer_revision": "64a8c8eab3e352a784c658aef62be1662607476f",
+  "training_config": {
+    "batch_size": 8,
+    "dual_collision_rows": 0,
+    "epochs": 3,
+    "learning_rate": 5e-06,
+    "replay_rows": 0,
+    "rows": 7440,
+    "seed": 20260914,
+    "weight_decay": 0.01
+  },
   "training_lineage": {
-    "combined_exposure": 418000,
-    "run_fingerprint": "96c110801753166276df98ca9d65406f161009b8e981c03ca8cca94a9fb489fe"
+    "frozen_tests_used_for_selection": false,
+    "selected_epoch": 3,
+    "selection_basis": "dev_only",
+    "starting_artifact_sha256": "119c0f19767b61446e04da1f8f01a001edf97a47a66965e7146db2483b4937a1",
+    "training_data_sha256": "ae5ec34e9ff02685212ef6d6702003b51161ad439d6bc2301b55a98f7392ff95",
+    "training_rows": 7440
   }
 }