Add README: product/launch tone, model family + lineage, compact results

README.md

@@ -0,0 +1,190 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen3-4B-Base
+library_name: pytorch
+tags:
+  - decision-model
+  - system-one
+  - calibrated-probabilities
+  - typed-decisions
+  - jev-like
+pipeline_tag: text-classification
+datasets:
+  - stanfordnlp/snli
+  - nyu-mll/multi_nli
+  - facebook/anli
+  - google/boolq
+  - cais/mmlu
+  - allenai/ai2_arc
+  - allenai/openbookqa
+  - tau/commonsense_qa
+  - allenai/sciq
+  - allenai/qasc
+  - jeggers/logiqa2_formatted
+  - deepmind/aqua_rat
+  - openlifescienceai/medmcqa
+  - cua-ai/cua-s1-forms
+  - dwidlee/systemone-lite-general
+  - MagaBitmex/jev-4b-distill-data
+  - Zhengping/UNLI
+  - metaeval/ambient
+---
+
+# typical-medium
+
+**Typical is an open System-One decision model family, built by OzLabs.**
+
+Give it a state — a support ticket, a document, a form, a log line, any text — a question with its
+rubric, and a label set defined at runtime, and it returns a calibrated probability over those
+labels plus an explicit abstain option. One forward pass, no generated text to parse, about 57 ms
+per decision on an H100. Every decision is one of three typed primitives: **Choice** (categorical),
+**Score** (ordinal), **Noul** (yes/no, exactly order-invariant).
+
+`typical-medium` is **Release 1 at 4B** (checkpoint `tm1b`): the same recipe as
+[`typical-small`](https://huggingface.co/OzLabs/typical-small) — dedicated Score and Noul heads, the
+hard curriculum, 1,024-token states — on a larger backbone. It is the knee of capability per
+millisecond in the release ladder: +11 points of JevBench standard and +11 points of MMLU-Pro
+among-K over `typical-small`, at 1.25x the per-decision latency.
+
+## Why it's different
+
+- **The decision is read directly from the model's own candidate-conditioned computation.** The
+  state and candidates are encoded once; the answer is read off that computation instead of being
+  sampled one token at a time.
+- **The state is KV-cached**, so asking many questions about one state is cheap — each additional
+  question is a short suffix against a cache that's already built.
+- **Latency is flat up to ~64 options** (57 ms at K = 2, 58 ms at K = 32); 96 ms at K = 256 — the
+  state dominates until the rendered option list does.
+- **Abstention is a first-class output**, not a parsed refusal: every decision carries a calibrated
+  P(∅) alongside the label probabilities.
+- **Probabilities, not verbalized confidence.** No "I'm 80% sure" string to parse — an actual
+  calibrated number.
+
+## Model family
+
+| | [`typical-small-preview`](https://huggingface.co/OzLabs/typical-small-preview) | [`typical-small`](https://huggingface.co/OzLabs/typical-small) | **`typical-medium`** |
+|---|---|---|---|
+| Backbone | Qwen3-1.7B-Base | Qwen3-1.7B-Base | Qwen3-4B-Base |
+| Tap layer | 20 / 28 | 20 / 28 | 26 / 36 |
+| Per-decision latency, K=2 (one H100) | 45 ms | 45 ms | 57 ms |
+| Peak memory, K=2 | 6.4 GB | 6.4 GB | 14.6 GB |
+| JevBench standard / easy / hard | .750 / 1.00 / .387 | .694 / 1.00 / .432 | .806 / 1.00 / .423 |
+| MMLU-Pro among-K | .330 | .343 | .458 |
+| Held-out workflow noul / score | .699 / .498 | .715 / .520 | .811 / .528 |
+| Calibration (typed-decisions NLL) | 2.06 | 1.25 | **1.18** |
+
+Full ablations and matched-baseline tables: REPORT.md §3af.
+
+## Results
+
+| | `typical-medium` (`tm1b`) |
+|---|---|
+| SNLI / MNLI / BoolQ / ANLI | .909 / .861 / .843 / .544 |
+| CLINC-150 / TREC-fine / HWU64 / 20NG | .847 / .414 / .769 / .588 |
+| Held-out workflow: noul / score / style | .811 / .528 / .859 |
+| External floor: PagerDuty (floor .792) | **.838** |
+| JevBench standard / easy / hard (Brier std / hard; ECE std) | .806 / 1.00 / .423 (.30 / .77; **.09**) |
+
+**JevBench standard, same 231 public ids, next to the open field:**
+
+| jeff (GLiFormer 400M) | Laya (ModernBERT) | **typical-medium** | open-alternative-jev (Qwen3.5-4B) | system-one-open (Gemma E2B LoRA) |
+|---|---|---|---|---|
+| .750 | .694 | **.806** | .833 | .931 |
+
+Public-subset per-item comparison on the 231 public ids; not a ranked leaderboard entry.
+
+## Quickstart
+
+```bash
+huggingface-cli download OzLabs/typical-medium --include "inference/*" --local-dir typical-medium
+pip install -r typical-medium/inference/requirements.txt
+```
+
+```python
+import sys; sys.path.insert(0, "typical-medium/inference")
+from typical import Typical
+
+m = Typical.from_pretrained("OzLabs/typical-medium", device="auto")
+
+ticket = ("Customer reports repeated 500 errors on checkout after the last deploy; "
+          "asking for a refund.")
+m.choice(ticket, "Which team should handle this?", ["billing", "infra", "support", "security"])
+m.score(ticket, "How severe is this ticket?", ["1", "2", "3", "4"])
+m.noul(ticket, "Does this ticket violate the refund-abuse policy (3+ refunds in 30 days)?")
+```
+
+Verified: this exact sequence (download, install, run) works on CPU (checked against `typical-small`'s
+identical inference package; `typical-medium` uses the same code path on a larger backbone).
+
+`choice`/`score` return `{label: probability, ...}` plus `p_null`; `noul` returns `P(yes)`. `decide`
+mirrors the internal `PCDMDecider.decide` exactly, including the runtime block (latency, `p_null`,
+truncation flags) — see `inference/README.md` for the full API and `inference/example.py` for a
+runnable script.
+
+## Training recipe
+
+- **Data mix** E .40 / K .15 / W .35 / U .10 of every batch (evidence, knowledge-MCQ, rubric-
+  conditioned workflow, soft-target uncertainty), plus 15–20% null augmentation on W rows
+  (`--null_aug W:0.20`) so the model learns to abstain, not just choose.
+- **1,024-token decision states**, so policy documents and multi-hop states are in-distribution.
+- **Typed heads:** Score is K-way Choice trained with ordinal-smoothed targets (τ = 0.7, zero head
+  code); Noul is a dedicated Bernoulli head, routed per row — only rows whose candidates are exactly
+  `{yes, no}` use it, every other 2-way set scores as ordinary Choice.
+- **12,000 steps, effective batch 64** (`bs 64`, `grad_accum 4`), LoRA rank 16 on the top 8 layers of
+  a 26-of-36-layer truncated backbone.
+- **Held-out by construction:** one rubric style and three whole domains of the hard curriculum
+  (`data_wh`), plus temporal/numeric/probability/trade-off composition (level 7), are eval-only —
+  never trained on.
+
+**Data sources and licenses:**
+
+| Source | Role | License |
+|---|---|---|
+| SNLI, MultiNLI, ANLI, BoolQ, MMLU-aux, ARC, OpenBookQA, CommonsenseQA, SciQ, QASC, LogiQA2, AQuA-RAT, MedMCQA | evidence + knowledge-MCQ | permissive research licenses (CC-BY-SA / MIT / Apache-2.0 family) |
+| `cua-ai/cua-s1-forms` | workflow (GUI-form synth) | MIT |
+| `dwidlee/systemone-lite-general` | workflow (rule-labeled typed decisions) | MIT |
+| `MagaBitmex/jev-4b-distill-data` | workflow (programmatic gold) | Apache-2.0 |
+| in-repo rubric-conditioned generator (`data_wf`, `data_wf_long`) | workflow | generated, no external license |
+| in-repo rule-engine generator (`data_wh`, DecisionMix v2) | hard curriculum | generated, no external license |
+| `Zhengping/UNLI` (validation split) | soft-target uncertainty | MIT |
+| `metaeval/ambient` (ambiguous rows, trained on) | soft-target uncertainty | **license undeclared on its HF card — flagged, not asserted** |
+| `metaeval/chaos-mnli-ambiguity` (eval-only, never trained on) | soft-target uncertainty | **license undeclared on its HF card — flagged, not asserted** |
+
+Evaluation-only, never trained on: JevBench (public subset), `typed-decisions`, PagerDuty, tree-choice,
+jevlogs (Loghub research license, caveated), Mind2Web.
+
+## Limitations
+
+- Hard tier is .423 (chance .336, SE ≈ .045 at n = 111) — the least reliable tier; the open field's
+  best is .73. The long-policy slice specifically is .21 — the weakest single family.
+- TREC-fine regresses 10 points vs. `ladder_4b`, the same backbone trained without typed heads or
+  DecisionMix v2 (K = 50 fine-grained topics) — the one clear regression at this size.
+- Held-out rubric styles and Noul accuracy give back a few points vs. that same untyped 4B baseline,
+  in exchange for the calibration and external-floor gains above.
+- Inference code is included in this repo; training code release to follow.
+
+## License
+
+Apache-2.0 (backbone `Qwen/Qwen3-4B-Base` is Apache-2.0; this release's weights are a LoRA adapter
+plus small Choice/Score/Noul heads over that backbone). Training-data licenses above; JevBench
+numbers are a public-subset run against `fstandhartinger/jevbench` v1.2.1, not a submitted or ranked
+leaderboard entry.
+
+## Files in this repository
+
+- `best.pt` — this checkpoint (`tm1b`).
+- `results.json` / `eval_wf_full.json` / `probe_results.json` / `jevbench_summary.json` — full eval
+  suite, post-hoc held-out/external eval, probe results, JevBench run summary.
+- `inference/` — the self-contained `Typical` inference package (see Quickstart above).
+- `MANIFEST.json` — checksum + provenance record for this release.
+
+## Lineage
+
+[`typical-small-preview`](https://huggingface.co/OzLabs/typical-small-preview) (Release 0) →
+[`typical-small`](https://huggingface.co/OzLabs/typical-small) (Release 1 at 1.7B) →
+**`typical-medium`** (Release 1 at 4B, this repo).
+
+Coming: `typical-large` — a 14B ladder checkpoint scores JevBench .875/.468 (standard/hard), but
+probability calibration at that size is not yet fixed (§3ab). Trained, not released.
+
+Typical is built by [OzLabs](https://huggingface.co/OzLabs).