Add model card with benchmark results

README.md

@@ -14,7 +14,7 @@ pipeline_tag: text-classification
 
 # classone-gemma4-e2b — ClassOne System 1 Decision Model
 
-**[devops-thiago/classone-gemma4-e2b](https://huggingface.co/devops-thiago/classone-gemma4-e2b)** is an open-source **System 1 decision model** using the [ClassOne architecture](https://github.com/devops-thiago/class-one). The full fine-tuned backbone ships directly in this repository — it loads as a single model, with no adapter and no separate base-model download.
+**[devops-thiago/classone-gemma4-e2b](https://huggingface.co/devops-thiago/classone-gemma4-e2b)** is an open-source **System 1 decision model** built on top of [`google/gemma-4-E2B-it`](https://huggingface.co/google/gemma-4-E2B-it) using the [ClassOne architecture](https://github.com/devops-thiago/class-one).
 
 Instead of generating text token by token, ClassOne evaluates structured decisions in a **single forward pass**, returning typed, calibrated outputs with zero decoding overhead.
 
@@ -29,7 +29,7 @@ Evaluated across all 231 public tasks in [fstandhartinger/jevbench](https://gith
 | **Easy** | 48 | **100.0%** (48/48) | **0.0064** | **0.0009** | **44.8 ms** |
 | **Original** | 72 | **90.3%** (65/72) | **0.0972** | **0.1015** | **42.9 ms** |
 | **Hard** | 111 | **44.1%** (49/111) | 0.4681 | 0.4813 | **90.0 ms** |
-| **Overall Aggregate** | **231** | **70.1%** (162/231) | — | — | **44.8 ms** |
+| **Overall Aggregate** | **231** | **70.1%** (162/231) | - | - | **44.8 ms** |
 
 - **Easy Tier Sub-Breakdown:** Choice accuracy: **100.0%** (36/36); Noul policy accuracy: **100.0%** (12/12). Flawless 0.0064 ECE.
 - **Original Tier Sub-Breakdown:** Noul accuracy: **100.0%** (24/24); Score rubrics: **100.0%** (12/12); Choice accuracy: **80.6%** (29/36).
@@ -49,6 +49,14 @@ Evaluated across the 10 core AI alignment failure modes (arXiv:2609.29429):
 | **Bias** | 9 | **0.525** | **55.6%** | 0.2141 | 217.9 ms |
 | **Overall Balanced Accuracy** | **100** | **0.505** | **56.2%** | **0.1944** | **199.2 ms** |
 
+### 3. Agentic Tool-Routing & Adversarial Safety Suites (2026 Benchmarks)
+
+| Benchmark Suite | Split / Samples | Accuracy | Secondary Metric | Calibration ECE | Latency (p50) |
+|---|---|---|---|---|---|
+| **WildGuardTest** (`walledai/WildGuardTest`) | `test` (100 stratified) | **72.00%** (72.04% BalAcc) | **0.7993 AUROC** (**78.6%** Vanilla, **72.9%** Benign Spec) | **0.1597** | 43.5 ms |
+| **Deepset Prompt Injections** (`deepset/prompt-injections`) | `test` (116 full) | **50.86%** (59/116) | **0.6214 AUROC** (**75.0%** Injection Recall) | 0.3412 | 41.6 ms |
+| **NVIDIA When2Call** (`nvidia/When2Call`) | `mcq` (100 stratified) | **35.00%** | **85.3%** `tool_call` (29/34) | 0.5376 | 106.8 ms |
+
 ### 3. Edge vs Cloud Latency (ClassOne vs TypeSafe Jev API)
 
 Measured against TypeSafe AI's Jev (v1.13) cloud API:
@@ -74,6 +82,7 @@ pip install classone
 ```python
 import torch
 from huggingface_hub import hf_hub_download
+from peft import PeftModel
 from transformers import AutoTokenizer
 
 from classone.modeling.modeling_classone import ClassOneModel
@@ -82,15 +91,16 @@ from classone.tokenizer import ClassOnePromptBuilder
 
 REPO_ID = "devops-thiago/classone-gemma4-e2b"
 
-# 1. Load the ClassOne model (weights + tokenizer are fully self-contained here)
-tokenizer = AutoTokenizer.from_pretrained(REPO_ID)
+# 1. Load the backbone and attach the published LoRA adapter
+tokenizer = AutoTokenizer.from_pretrained("google/gemma-4-E2B-it")
 builder = ClassOnePromptBuilder(tokenizer)
 model = ClassOneModel.from_backbone(
-    base_model_name_or_path=REPO_ID,
+    base_model_name_or_path="google/gemma-4-E2B-it",
     tokenizer=tokenizer,
     device="cuda",
     torch_dtype=torch.float16,
 )
+model.backbone = PeftModel.from_pretrained(model.backbone, REPO_ID, subfolder="lora_backbone")
 
 # 2. Load the trained decision heads
 heads = torch.load(hf_hub_download(REPO_ID, "classone_heads.pt"), map_location="cuda")
@@ -125,11 +135,9 @@ print("Anger score:   ", results["anger"].score)
 
 | File | Description |
 |---|---|
-| `model.safetensors` (sharded) | Merged ClassOne backbone weights |
-| `config.json` | Model configuration |
-| `tokenizer.json`, `tokenizer_config.json` | Tokenizer, including ClassOne delimiter tokens |
 | `classone_heads.pt` | Trained Noul / Choice / Score head weights + calibrated temperatures |
-| `lora_backbone/` | LoRA adapter (r=16, α=32) that produced the merged weights |
+| `lora_backbone/adapter_model.safetensors` | LoRA adapter weights (r=16, α=32) |
+| `lora_backbone/adapter_config.json` | LoRA config (target modules, rank, etc.) |
 
 ## Citation