Add model card with benchmark results

README.md

@@ -14,7 +14,7 @@ pipeline_tag: text-classification
 
 # classone-qwen3.5-4b — ClassOne System 1 Decision Model
 
-**[devops-thiago/classone-qwen3.5-4b](https://huggingface.co/devops-thiago/classone-qwen3.5-4b)** is an open-source **System 1 decision model** using the [ClassOne architecture](https://github.com/devops-thiago/class-one). The full fine-tuned backbone ships directly in this repository — it loads as a single model, with no adapter and no separate base-model download.
+**[devops-thiago/classone-qwen3.5-4b](https://huggingface.co/devops-thiago/classone-qwen3.5-4b)** is an open-source **System 1 decision model** built on top of [`Qwen/Qwen3.5-4B`](https://huggingface.co/Qwen/Qwen3.5-4B) using the [ClassOne architecture](https://github.com/devops-thiago/class-one).
 
 Instead of generating text token by token, ClassOne evaluates structured decisions in a **single forward pass**, returning typed, calibrated outputs with zero decoding overhead.
 
@@ -26,35 +26,35 @@ Evaluated across all 231 public tasks in [fstandhartinger/jevbench](https://gith
 
 | Tier | Tasks | Accuracy | ECE | Brier Score | Median Latency (p50) |
 |---|---|---|---|---|---|
-| **Easy** | 48 | **100.0%** (48/48) | **0.0001** | **0.0000** | **63.4 ms** |
-| **Original** | 72 | **95.8%** (69/72) | **0.0458** | **0.0377** | **61.1 ms** |
-| **Hard** | 111 | **47.7%** (53/111) | 0.4520 | 0.4201 | **184.5 ms** |
-| **Overall Aggregate** | **231** | **73.6%** (170/231) [4B RECORD] | — | — | **61.1 ms** |
+| **Easy** | 48 | **100.0%** (48/48) | **0.0257** | **0.0094** | **63.3 ms** |
+| **Original** | 72 | **95.8%** (69/72) | **0.0481** | **0.0352** | **62.9 ms** |
+| **Hard** | 111 | **52.2%** (58/111) | 0.3099 | 0.3424 | **211.2 ms** |
+| **Overall Aggregate** | **231** | **75.8%** (175/231) | — | — | **63.3 ms** |
 
-- **Original Tier Sub-Breakdown:** Choice accuracy: **97.2%** (35/36); Score rubrics: **100.0%** (12/12); Noul accuracy: **91.7%** (22/24).
-- **Easy Tier Sub-Breakdown:** Choice accuracy: **100.0%** (36/36); Noul policy accuracy: **100.0%** (12/12).
-- **Hard Tier Sub-Breakdown:** Choice accuracy: **46.3%** (31/67); Noul accuracy: **50.0%** (19/38); Score: **50.0%** (3/6).
+- **Easy Tier Sub-Breakdown:** Choice: **100.0%** (36/36); Noul policy: **100.0%** (12/12). Flawless 100%.
+- **Original Tier Sub-Breakdown:** Noul: **100.0%** (24/24); Score rubrics: **100.0%** (12/12); Choice: **91.7%** (33/36).
+- **Hard Tier Sub-Breakdown:** Choice: **52.2%** (35/67); Noul: **52.6%** (20/38); Score: **50.0%** (3/6).
 
 ### 2. Hugging Face Decision Index (50 Public Multi-Domain Tasks)
 
-| Domain | Tasks | Accuracy (%) |
-|---|---|---|
-| **Knowledge** | 10 | **100.0%** (10/10) |
-| **Language** | 10 | **100.0%** (10/10) |
-| **Retrieval** | 10 | **100.0%** (10/10) |
-| **Rubric** | 10 | **90.0%** (9/10) |
-| **Tools** | 10 | **100.0%** (10/10) |
-| **Overall Decision Index** | **50** | **98.0%** (49/50) |
+| Domain | Weight | Accuracy (%) | Score |
+|---|---|---|---|
+| **Knowledge & Verification** | 25.8% | **100.0%** (10/10) | 100.00 pts |
+| **Language & Policy Contracts** | 25.8% | **100.0%** (10/10) | 100.00 pts |
+| **Retrieval & Relevance Triage** | 20.0% | **100.0%** (10/10) | 100.00 pts |
+| **Tools & Execution Guardrails** | 18.3% | **100.0%** (10/10) | 100.00 pts |
+| **Rubric & Quality Scoring** | 10.1% | **90.0%** (9/10) | 90.00 pts |
+| **Overall Decision Index** | **100.0%** | **98.0%** (49/50) | **98.99 / 100.00** (0.0317 ECE) |
 
 ### 3. RLCDAlignBench Alignment & Safety Evaluation (100 Instances)
 
 | Failure Mode / Axis | Samples (N) | AUROC | Accuracy (%) | ECE | Latency (p50) |
 |---|---|---|---|---|---|
-| **Power Seeking** | 6 | **0.778** | **83.3%** | 0.1475 | 441.5 ms |
-| **Faithfulness** | 9 | **0.850** | **66.7%** | 0.1956 | 433.4 ms |
-| **Refusal (Jailbreaks)** | 11 | **0.733** | **72.7%** | **0.0753** | 570.5 ms |
-| **Honesty (Deception)** | 11 | **0.733** | **72.7%** | 0.2583 | 453.5 ms |
-| **Overall AUROC** | **100** | **0.604** | **55.3%** | **0.2553** | **410.9 ms** |
+| **Power Seeking** | 6 | **0.889** | **83.3%** | 0.2008 | 520.1 ms |
+| **Faithfulness** | 9 | **0.850** | **77.8%** | 0.2895 | 513.4 ms |
+| **Honesty (Deception)** | 11 | **0.833** | **63.6%** | 0.2482 | 536.0 ms |
+| **Refusal (Jailbreaks)** | 11 | **0.600** | **45.5%** | 0.3167 | 670.6 ms |
+| **Overall Balanced Accuracy** | **100** | **0.562** | **51.5%** | **0.3101** | **487.9 ms** |
 
 ### 3. Edge vs Cloud Latency (ClassOne vs TypeSafe Jev API)
 
@@ -81,6 +81,7 @@ pip install classone
 ```python
 import torch
 from huggingface_hub import hf_hub_download
+from peft import PeftModel
 from transformers import AutoTokenizer
 
 from classone.modeling.modeling_classone import ClassOneModel
@@ -89,15 +90,16 @@ from classone.tokenizer import ClassOnePromptBuilder
 
 REPO_ID = "devops-thiago/classone-qwen3.5-4b"
 
-# 1. Load the ClassOne model (weights + tokenizer are fully self-contained here)
-tokenizer = AutoTokenizer.from_pretrained(REPO_ID)
+# 1. Load the backbone and attach the published LoRA adapter
+tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-4B")
 builder = ClassOnePromptBuilder(tokenizer)
 model = ClassOneModel.from_backbone(
-    base_model_name_or_path=REPO_ID,
+    base_model_name_or_path="Qwen/Qwen3.5-4B",
     tokenizer=tokenizer,
     device="cuda",
     torch_dtype=torch.float16,
 )
+model.backbone = PeftModel.from_pretrained(model.backbone, REPO_ID, subfolder="lora_backbone")
 
 # 2. Load the trained decision heads
 heads = torch.load(hf_hub_download(REPO_ID, "classone_heads.pt"), map_location="cuda")
@@ -132,11 +134,9 @@ print("Anger score:   ", results["anger"].score)
 
 | File | Description |
 |---|---|
-| `model.safetensors` (sharded) | Merged ClassOne backbone weights |
-| `config.json` | Model configuration |
-| `tokenizer.json`, `tokenizer_config.json` | Tokenizer, including ClassOne delimiter tokens |
 | `classone_heads.pt` | Trained Noul / Choice / Score head weights + calibrated temperatures |
-| `lora_backbone/` | LoRA adapter (r=16, α=32) that produced the merged weights |
+| `lora_backbone/adapter_model.safetensors` | LoRA adapter weights (r=16, α=32) |
+| `lora_backbone/adapter_config.json` | LoRA config (target modules, rank, etc.) |
 
 ## Citation