Update model card metadata and benchmark provenance

README.md

@@ -1,6 +1,6 @@
 ---
 license: apache-2.0
-base_model: google/gemma-4-E2B-it
+base_model: google/gemma-4-e2b-it
 tags:
   - decision-model
   - system-1
@@ -14,30 +14,25 @@ pipeline_tag: text-classification
 
 # classone-gemma4-e2b — ClassOne System 1 Decision Model
 
-**[devops-thiago/classone-gemma4-e2b](https://huggingface.co/devops-thiago/classone-gemma4-e2b)** is an open-source **System 1 decision model** using the [ClassOne architecture](https://github.com/devops-thiago/class-one). The full fine-tuned backbone ships directly in this repository — it loads as a single model, with no adapter and no separate base-model download.
+**[devops-thiago/classone-gemma4-e2b](https://huggingface.co/devops-thiago/classone-gemma4-e2b)** is an open-source **System 1 decision model** built on [`google/gemma-4-e2b-it`](https://huggingface.co/google/gemma-4-e2b-it) using the [ClassOne architecture](https://github.com/devops-thiago/class-one). The merged standalone backbone ships directly in this repository — it loads as a single model, with no adapter and no separate base-model download.
 
 Instead of generating text token by token, ClassOne evaluates structured decisions in a **single forward pass**, returning typed, calibrated outputs with zero decoding overhead.
 
-## Benchmark Results
+## Self-Measured Benchmark Results
 
-### 1. JevBench Public Multi-Tier Benchmark (231 Public Tasks)
+> **Self-Measured Evaluation Provenance:** Executed locally on `2x NVIDIA RTX 5060 Ti 16 GB` (Date: `2026-11-13`, Commit: `c375207`). See [Official Results (Benchmark Heaven)](#official-results-benchmark-heaven) below for third-party sealed evaluation.
 
-Evaluated across all 231 public tasks in [fstandhartinger/jevbench](https://github.com/fstandhartinger/jevbench):
+> **Tokenizer Compatibility Note (`commit 86988ca` / SDK `v0.2.0–v0.2.3`):** The standalone weights in this repository were trained with custom delimiter tokens (`<|state_start|>=262144..262155`) serialized in `tokenizer.json`. Later SDK releases (`v0.2.0–v0.2.3`) prioritize Gemma 4 `<unused0>..` reserved tokens when present; with native `<|state_start|>=262144` formatting, these weights + rolled-back `classone_heads.pt` (`F6A6E423...`) achieve **69.3%** (160/231) on JevBench (whereas default `<unused0>` formatting scores **58.4%** / 135/231).
 
-| Tier | Tasks | Accuracy | ECE | Brier Score | Median Latency (p50) |
-|---|---|---|---|---|---|
-| **Easy** | 48 | **100.0%** (48/48) | **0.0064** | **0.0009** | **44.8 ms** |
-| **Original** | 72 | **90.3%** (65/72) | **0.0972** | **0.1015** | **42.9 ms** |
-| **Hard** | 111 | **44.1%** (49/111) | 0.4681 | 0.4813 | **90.0 ms** |
-| **Overall Aggregate** | **231** | **70.1%** (162/231) | - | - | **44.8 ms** |
-
-- **Easy Tier Sub-Breakdown:** Choice accuracy: **100.0%** (36/36); Noul policy accuracy: **100.0%** (12/12). Flawless 0.0064 ECE.
-- **Original Tier Sub-Breakdown:** Noul accuracy: **100.0%** (24/24); Score rubrics: **100.0%** (12/12); Choice accuracy: **80.6%** (29/36).
-- **Hard Tier Sub-Breakdown:** Noul policy compliance: **52.6%** (20/38); Choice accuracy: **43.3%** (29/67).
+### 1. JevBench Public Multi-Tier Suite (231 Public Tasks, Self-Measured Standalone)
 
-### 2. RLCDAlignBench Alignment & Safety Evaluation (100 Instances)
+| Evaluation Regime | Overall (231) | Easy (48) | Original (72) | Hard (111) |
+|---|---|---|---|---|
+| **Current Hub Weights + Rolled-Back Heads (`F6A6E423...`, Native `<\|state_start\|>`)** | **69.3%** (160/231) | **93.8%** (45/48) | **88.9%** (64/72) | **45.9%** (51/111) |
+| **Original Milestone Snapshot (`best_composite_70pct_milestone`, 2026-10-06)** | **70.1%** (162/231) | **100.0%** (48/48, 0.0064 ECE) | **90.3%** (65/72, 0.0972 ECE) | **44.1%** (49/111) |
+| **Current Hub Weights under SDK `v0.2.0–v0.2.3` (`<unused0>`)** | 58.4% (135/231) | 91.7% (44/48) | 70.8% (51/72) | 36.0% (40/111) |
 
-Evaluated across the 10 core AI alignment failure modes (arXiv:2609.29429):
+### 2. RLCDAlignBench Alignment & Safety Suite (100 Instances, 2026-10-06 Snapshot)
 
 | Failure Mode / Axis | Samples (N) | AUROC | Accuracy (%) | ECE | Latency (p50) |
 |---|---|---|---|---|---|
@@ -49,7 +44,7 @@ Evaluated across the 10 core AI alignment failure modes (arXiv:2609.29429):
 | **Bias** | 9 | **0.525** | **55.6%** | 0.2141 | 217.9 ms |
 | **Overall Balanced Accuracy** | **100** | **0.505** | **56.2%** | **0.1944** | **199.2 ms** |
 
-### 3. Agentic Tool-Routing & Adversarial Safety Suites (2026 Benchmarks)
+### 3. Tool-Routing & Safety Guardrail Suites (Self-Measured)
 
 | Benchmark Suite | Split / Samples | Accuracy | Secondary Metric | Calibration ECE | Latency (p50) |
 |---|---|---|---|---|---|
@@ -57,11 +52,15 @@ Evaluated across the 10 core AI alignment failure modes (arXiv:2609.29429):
 | **Deepset Prompt Injections** (`deepset/prompt-injections`) | `test` (116 full) | **50.86%** (59/116) | **0.6214 AUROC** (**75.0%** Injection Recall) | 0.3412 | 41.6 ms |
 | **NVIDIA When2Call** (`nvidia/When2Call`) | `mcq` (100 stratified) | **35.00%** | **85.3%** `tool_call` (29/34) | 0.5376 | 106.8 ms |
 
-### 3. Edge vs Cloud Latency (ClassOne vs TypeSafe Jev API)
+## Official Results (Benchmark Heaven)
+
+Official third-party leaderboards are hosted at **[Benchmark Heaven](https://benchmarkheaven.com/jev-models)** (see e.g. [`classone-qwen3.5-9b`](https://benchmarkheaven.com/jev-models/classone-qwen3.5-9b) for `v1.8.8` scoring). Official Benchmark Heaven scores differ from local self-measurements because the official harness evaluates private **sealed items** in addition to the public suite, applies a public/sealed generalization gap penalty, enforces a strict 10-second cutoff rule, and measures latency/cost on server-grade GPUs rather than local workstation hardware (`2x NVIDIA RTX 5060 Ti 16 GB`).
+
+## Edge vs Cloud Latency (Self-Measured Comparison)
 
-Measured against TypeSafe AI's Jev (v1.13) cloud API:
+Measured against a remote cloud LLM decision API round-trip:
 - **ClassOne (Local RTX 5060 Ti):** **52.49 ms** mean latency (19.1 req/s, $0.00 inference cost, 100% private)
-- **TypeSafe Jev (Cloud API):** **329.90 ms** mean latency (3.0 req/s)
+- **Cloud API Baseline:** **329.90 ms** mean latency (3.0 req/s)
 - **Edge Speedup:** **6.3× faster** than cloud API round-trip latency
 
 ## Decision Primitives
@@ -70,8 +69,7 @@ Measured against TypeSafe AI's Jev (v1.13) cloud API:
 - **`Choice`** — Categorical selection over 2–255 dynamic options with full probability distribution
 - **`Score`** — Continuous ordinal rubric rating over 2–10 levels (expected value)
 
-All outputs are calibrated with a combined NLL + normalized Brier loss.
-Post-hoc temperature calibration achieves **ECE = 0.034** (down from 0.178).
+All outputs are calibrated with a combined NLL + normalized Brier loss and post-hoc temperature scaling.
 
 ## Quickstart
 
@@ -102,9 +100,9 @@ model = ClassOneModel.from_backbone(
 
 # 2. Load the trained decision heads
 heads = torch.load(hf_hub_download(REPO_ID, "classone_heads.pt"), map_location="cuda")
-model.noul_head.load_state_dict(heads["noul_head"])
-model.choice_head.load_state_dict(heads["choice_head"])
-model.score_head.load_state_dict(heads["score_head"])
+model.noul_head.load_state_dict(heads["noul_head"], strict=False)
+model.choice_head.load_state_dict(heads["choice_head"], strict=False)
+model.score_head.load_state_dict(heads["score_head"], strict=False)
 model.eval()
 
 # 3. Pack state + questions and run a single forward pass
@@ -133,11 +131,10 @@ print("Anger score:   ", results["anger"].score)
 
 | File | Description |
 |---|---|
-| `model.safetensors` (sharded) | Merged ClassOne backbone weights |
+| `model.safetensors` (or sharded) | Merged ClassOne standalone backbone weights |
 | `config.json` | Model configuration |
-| `tokenizer.json`, `tokenizer_config.json` | Tokenizer, including ClassOne delimiter tokens |
+| `tokenizer.json`, `tokenizer_config.json` | Tokenizer with ClassOne delimiter tokens |
 | `classone_heads.pt` | Trained Noul / Choice / Score head weights + calibrated temperatures |
-| `lora_backbone/` | LoRA adapter (r=16, α=32) that produced the merged weights |
 
 ## Citation
 
@@ -152,5 +149,5 @@ print("Anger score:   ", results["anger"].score)
 
 ## Attribution & Legal
 
-- Derived from [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it) (Google) — Apache License 2.0
+- Derived from [google/gemma-4-e2b-it](https://huggingface.co/google/gemma-4-e2b-it) (Google) — Apache License 2.0
 - Architecture & training code: [devops-thiago/class-one](https://github.com/devops-thiago/class-one) — Apache 2.0