Model card: canonical macro-averaged results, output schema section, corrected method description and chart

README.md

@@ -1,6 +1,8 @@
 ---
 license: gemma
 base_model: google/gemma-3-1b-it
+datasets:
+  - zero-proof-ai/ecommerce-intent
 library_name: peft
 pipeline_tag: text-classification
 tags:
@@ -18,7 +20,7 @@ language:
 
 **Frontier-approaching e-commerce intent detection in a 1B model.** Given a live customer conversation, `zeroproof-ecommerce-1b` returns a single JSON object with the customer's payment intent and its structured details. It runs inline on live traffic, at a cost and latency where calling a frontier model is not an option.
 
-On a hard, held-out benchmark it reaches **70% exact-match core-intent accuracy, 88% of GPT-5's**, at roughly **1/1000th the parameters** and **$0.18 per 1M tokens**.
+On a hard, held-out benchmark it reaches **70 macro-averaged intent-type accuracy, 88% of GPT-5's**, at **$0.18 per 1M output tokens** and about a second per response.
 
 ## Model Details
 
@@ -33,6 +35,7 @@ On a hard, held-out benchmark it reaches **70% exact-match core-intent accuracy,
 ### Model Sources
 
 - **Repository:** https://huggingface.co/zero-proof-ai/zeroproof-ecommerce-1b
+- **Training data:** https://huggingface.co/datasets/zero-proof-ai/ecommerce-intent
 
 ## Uses
 
@@ -46,7 +49,7 @@ English-language e-commerce and payments only. It is a narrow intent classifier,
 
 ## Bias, Risks, and Limitations
 
-Trained and evaluated on synthetic role-play conversations distilled from frontier models under a locked labeling policy; live-traffic distribution may differ. Accuracy is currently strongest on refunds, exchanges, transfers, and subscriptions; checkout and bill-pay are the focus of the current data round.
+Trained and evaluated on synthetic role-play conversations from a multi-agent marketplace simulation, labeled in a separate model pass under a locked labeling policy; labels are model-assigned and carry residual noise, and live-traffic distribution may differ. Accuracy is currently strongest on refunds, exchanges, and transfers; checkout and bill-pay are the focus of the current data round.
 
 ### Recommendations
 
@@ -62,13 +65,26 @@ model = AutoPeftModelForCausalLM.from_pretrained("zero-proof-ai/zeroproof-ecomme
 tok   = AutoTokenizer.from_pretrained("zero-proof-ai/zeroproof-ecommerce-1b")
 ```
 
-Use greedy decoding (`temperature=0`). The system prompt and per-intent detail schemas ship in the repo.
+Prompt it with the system prompt shipped in `prompt_template.txt` plus the conversation so far, and decode greedily (`temperature=0`).
+
+### Output format
+
+The model returns one JSON object per message:
+
+```json
+{"intent_detected": true, "core_type": "reverse",
+ "details": {"action": "refund", "merchant": "Amazon", "amount": 81.40, "reason_code": "scam"},
+ "confidence": 0.85, "reason": "Customer wants a refund on a suspicious charge.",
+ "source_message_seqs": [2]}
+```
+
+`core_type` is one of `spend`, `send`, `exchange`, `recur`, `bill`, `reverse`, or `none`, and each type carries its own `details` fields.
 
 ## Training Details
 
 ### Training Data
 
-The differentiator is the data. ZeroProof builds e-commerce intent training data as **randomized conversational simulations**: role-played customers with independently sampled personas, tones, financial situations, life stages, devices, and behaviors, including adversarial actors, distilled from frontier models under a locked labeling policy. Generation is label-blind (the generating models never see the intent schema), and every candidate passes a structural data gate: deduplicated by message signature, with zero train/eval leakage.
+The differentiator is the data. ZeroProof builds e-commerce intent training data as **randomized conversational simulations**: role-played customers with independently sampled personas, tones, financial situations, life stages, devices, and behaviors, including adversarial actors, simulated turn by turn between two independently drawn models. Generation is label-blind (the generating models never see the intent schema), labels are assigned in a separate pass, and every candidate passes a structural data gate: deduplicated by message signature, with zero train/eval leakage.
 
 ### Training Procedure
 
@@ -84,15 +100,16 @@ The differentiator is the data. ZeroProof builds e-commerce intent training data
 
 ### Testing Data, Factors & Metrics
 
-Held-out eval of 1,977 conversations, zero train/eval leakage, macro-averaged (equal weight per intent), scored by exact match against gold. The frontier panel is scored on a 412-row balanced subset of the same eval.
+Held-out eval of 1,977 conversations, zero train/eval leakage, macro-averaged (equal weight per intent), scored deterministically against gold: intent type by exact match, order details as the fraction of gold fields reproduced (partial credit per field). The frontier panel is scored on a 412-row balanced subset of the same eval.
 
 ### Results
 
-![ZeroProof e-commerce intent evaluation: accuracy, per-action accuracy, and cost to serve](rubric_short.png)
+![ZeroProof e-commerce intent evaluation: accuracy, per-action accuracy, and cost to serve](1b-v1.png)
 
-- Fine-tuning takes core-intent accuracy from **12% to 70%**, landing at **88% of GPT-5** and within 10 points of Opus 4.8.
-- On structured detail extraction it **edges GPT-5** (44% vs 41%).
-- No easy rows: every conversation is adversarial (negation, ambiguity) or long. The model holds **65%** on the hard slice and **78%** on the long slice, where the base model scores 18% and 11%.
+- Fine-tuning takes intent-type accuracy from **12 to 70** (macro), landing at **88% of GPT-5's 79.5**.
+- On structured detail extraction it **exceeds the full frontier panel**: 63 against Sonnet 5's 56, Opus 4.8's 53, and GPT-5's 41.
+- On the no-action boundary it errs toward inaction: **4.2%** of true no-action conversations are marked actionable, with remaining errors concentrated in the safe direction (missed actions, not invented ones).
+- No easy rows: every conversation is adversarial (negation, ambiguity) or long. The model holds **65** on the hard slice and **70** on the long slice, where the base scores 18 and 10.
 
 ## Technical Specifications