Model card: leaner client-facing rewrite, e-commerce positioning, rubric_short

README.md

@@ -14,48 +14,26 @@ language:
   - en
 ---
 
-# zeroproof-ecommerce-1b, v1.1
+# zeroproof-ecommerce-1b
 
-A **1B-parameter** model that detects and classifies **payment intent turn-by-turn, live during a conversation**. Given the dialogue so far, it emits a single JSON object: whether the user has an actionable payment intent yet, which of seven types it is, and the extracted details.
+**Frontier-approaching e-commerce intent detection in a 1B model.**
 
-It is a **QLoRA fine-tune of `gemma-3-1b-it`**. The point is not to beat frontier models, it's to get a large fraction of frontier accuracy on this narrow task at **~1/1000th the parameters**, cheap and fast enough to call on *every* turn of *every* conversation.
+`zeroproof-ecommerce-1b` reads a live customer conversation and returns a JSON object with the customer's payment intent and its structured details. It runs inline on live traffic, at a cost and latency where calling a frontier model is not an option.
 
-> **v1.1 · private preview.** Numbers are honest and reproducible; some intents are still weak (see Limitations) and are the target of the next data round.
+On a hard, held-out benchmark it reaches **70% exact-match core-intent accuracy, 88% of GPT-5's**, at roughly **1/1000th the parameters** and **$0.18 per 1M tokens**.
 
-## What it does
+Built by ZeroProof. Architecturally a QLoRA fine-tune of `gemma-3-1b-it`; the accuracy comes from our e-commerce intent dataset.
 
-Seven routing outcomes, each with its own detail schema:
+## Benchmarks
 
-`spend` · `send` · `exchange` · `recur` · `bill` · `reverse` · `none`
+![ZeroProof e-commerce intent evaluation: accuracy, per-action accuracy, and cost to serve](rubric_short.png)
 
-`none` is the safety default, the model returns it while the user is *thinking about* a purchase and only commits to an action intent once the user actually decides. That deferral behavior is the core of "turn-by-turn": the intent can flip from `none` to an action mid-conversation, on the turn the user commits.
+Held-out eval of 1,977 conversations, zero train/eval leakage, macro-averaged, exact-match against gold. The frontier panel is scored on a 412-row balanced subset of the same eval.
 
-## Results
-
-![Rubric, base vs SFT vs frontier](rubric.png)
-
-Held-out eval: **1,977 conversations**, zero train/eval leakage, **macro-averaged** (equal weight per intent, so `none` cannot inflate the score). Every model below is scored **identically**, exact-match against gold. Frontier panel is a 412-row balanced subset of the same eval.
-
-| Metric (macro) | gemma-1b | **zeroproof-ecommerce-1b** | GPT-5 | Sonnet 5 | Opus 4.8 |
-|---|---|---|---|---|---|
-| Valid JSON | 72.7% | **99.2%** | 100% | 96.1% | 100% |
-| Intent detected | 54.9% | **74.9%** | 85.8% | 89.9% | 92.4% |
-| **Correct core_type** | 12.1% | **70.0%** | 79.5% | 87.0% | 88.2% |
-| Details filled | 1.8% | **44.3%** | 41.4% | 55.7% | 53.2% |
-
-- Core-type accuracy goes **12% → 70%** with fine-tuning, reaching **~88% of GPT-5** and **~80% of Opus** on this task, at a fraction of the size, cost, and latency.
-- On **details extraction it matches GPT-5** (44% vs 41%).
-- The eval contains **no easy rows**, every conversation is hard (negation/ambiguity) or long (16+ turns). zeroproof-ecommerce-1b scores **65%** on hard and **78%** on long slices (base: 18% / 11%).
-
-Per-intent core_type (zeroproof-ecommerce-1b): `none` 95 · `exchange` 73 · `reverse` 73 · `send` 70 · `recur` 68 · `bill` 57 · `spend` 55.
-
-## Cost & latency (measured)
-
-Benchmarked on a **Modal L4 GPU** (~$0.80/hr) running `gemma-3-1b` + this LoRA adapter under vLLM (greedy, 256 max new tokens, 64 held-out prompts):
-
-- **$0.18 / 1M output tokens** at batched capacity (1,205 tok/s), roughly **55–140× cheaper** than frontier list prices ($10–25/1M).
-- **~1 s per turn** single-request latency (p50 984 ms, p95 1,191 ms).
-- Single-stream with the GPU idle between requests is pricier (~$1.96/1M); real cost sits between, depending on load.
+- Fine-tuning takes core-intent accuracy from **12% to 70%**, landing at **88% of GPT-5** and within 10 points of Opus 4.8.
+- On structured detail extraction it **edges GPT-5** (44% vs 41%).
+- No easy rows: every conversation is adversarial (negation, ambiguity) or long. The model holds **65%** on the hard slice and **78%** on the long slice, where the base model scores 18% and 11%.
+- **$0.18 per 1M output tokens** and **~1s per response**, 55 to 140x below frontier list prices.
 
 ## Usage
 
@@ -67,17 +45,21 @@ base = AutoModelForCausalLM.from_pretrained("google/gemma-3-1b-it")
 tok = AutoTokenizer.from_pretrained("google/gemma-3-1b-it")
 model = PeftModel.from_pretrained(base, "zero-proof-ai/zeroproof-ecommerce-1b")
 
-# Feed the conversation so far; the model emits one JSON object.
+# Feed the conversation so far; the model returns one JSON object per message.
 ```
 
-The system prompt and per-intent detail schemas are in the repo. Greedy decoding (`temperature=0`).
+The system prompt and detail schemas ship in the repo. Use greedy decoding (`temperature=0`).
+
+## Training and data
+
+A 4-bit QLoRA fine-tune of `gemma-3-1b-it` (via [unsloth](https://github.com/unslothai/unsloth)), one epoch, 12288-token context.
+
+The differentiator is the data. ZeroProof builds e-commerce intent training data as **randomized conversational simulations**: role-played customers with independently sampled personas, tones, financial situations, and life stages, including adversarial actors, distilled from frontier models under a locked labeling policy and passed through a structural data gate (deduplicated by message signature, zero train/eval leakage). This corpus, not the fine-tuning recipe, is where the accuracy comes from.
 
-## Training
+## Intended use
 
-- **Base:** `gemma-3-1b-it`, 4-bit QLoRA via [unsloth](https://github.com/unslothai/unsloth).
-- **Data:** 13,282 label-blind role-play conversations, relabeled under a locked written policy and validated by a data gate (deduplicated by message signature, zero train/eval leakage). Deliberately `none`-heavy to mirror real traffic; macro-averaging keeps the reported score honest.
-- **Recipe:** 1 epoch, effective batch 16, LR 2e-4, seq len 12288 (long-context conversations fit). Loss plateaus by ~step 800; held-out accuracy confirms no memorization.
+English-language e-commerce and payments support. Accuracy is currently strongest on refunds, exchanges, transfers, and subscriptions; checkout and bill-pay are the focus of the current data round.
 
 ## License
 
-Adapter released under the base model's **Gemma** license terms. Eval set open-sourced separately for reproducibility.
+Adapter released under the base model's **Gemma** license terms.