docs: add self-contained RLCD loss code, architecture lineage, and CMU benchmarks

README.md

@@ -1,5 +1,5 @@
 ---
-base_model: Qwen/Qwen3.5-9B-Base
+base_model: jaredpalmer/kev-9b
 library_name: peft
 pipeline_tag: text-classification
 tags:
@@ -34,34 +34,52 @@ metrics:
 <div align="center">
 
 [![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model%20Hub-blue?style=for-the-badge)](https://huggingface.co/gyung/Qwev-9B-RLCD)
-[![GitHub](https://img.shields.io/badge/GitHub-Jev--Qwen-181717?style=for-the-badge&logo=github)](https://github.com/gyunggyung/Jev-Qwen)
+[![Base Model](https://img.shields.io/badge/Base%20Model-jaredpalmer%2Fkev--9b-orange?style=for-the-badge)](https://huggingface.co/jaredpalmer/kev-9b)
 [![Paper](https://img.shields.io/badge/arXiv-2609.26550-B31B1B.svg?style=for-the-badge)](https://arxiv.org/html/2609.26550)
 [![License](https://img.shields.io/badge/License-Apache%202.0-green.svg?style=for-the-badge)](LICENSE)
 
 </div>
 
 > **"Accept When Confident, Escalate When Unsure."**  
-> **Qwev-9B-RLCD** is a non-autoregressive System 1 decision model aligned with **Reinforcement Learning from Calibrated Decisions (RLCD)**. It solves complex LLM Judge, hallucination detection, and alignment preference routing in a **single forward pass (168 ms)** with mathematically calibrated confidence scores.
+> **Qwev-9B-RLCD** is a non-autoregressive System 1 decision model aligned via **Reinforcement Learning from Calibrated Decisions (RLCD)** on top of [`jaredpalmer/kev-9b`](https://huggingface.co/jaredpalmer/kev-9b) (with a `Qwen/Qwen3.5-9B-Base` backbone). It solves complex LLM Judge, factuality/hallucination detection, and alignment preference decisions in a **single forward pass (168 ms)** with mathematically calibrated confidence scores.
+
+---
+
+## 🌳 Model Lineage & Tree
+
+```
+Qwen/Qwen3.5-9B-Base (9B Recurrent/DeltaNet Hybrid Backbone)
+       │
+       ▼
+jaredpalmer/kev-9b (Pointer Head SFT Adaptation)
+       │
+       ▼  [Aligned via RLCD Reinforcement Learning on NVIDIA A100-80GB]
+gyung/Qwev-9B-RLCD (Ours: Near-Zero Calibration Error & SOTA Accuracy)
+```
+
+- **Base Backbone**: [`Qwen/Qwen3.5-9B-Base`](https://huggingface.co/Qwen/Qwen3.5-9B-Base)
+- **Direct Parent Model**: [`jaredpalmer/kev-9b`](https://huggingface.co/jaredpalmer/kev-9b)
+- **Adaptation Mechanism**: Trainable LoRA Adapter + Pointer Softmax Readout Head (`head.pt`)
 
 ---
 
 ## 🌟 Key Highlights
 
 - 🚀 **1-Pass Non-Autoregressive Inference**: Zero token generation overhead. Decisions are made in **168.4 ms** (~11x faster than generative LLMs like GPT-6 Astra at 1,885 ms).
-- 🏆 **SOTA Decision Accuracy**:
+- 🏆 **SOTA Decision Accuracy (CMU Table 1 Protocol)**:
   - **RewardBench (400 samples)**: **99.2%** *(Outperforming CMU JEV 1.13: 92.2% & GPT-6 Astra: 93.5%)*
   - **HaluEval (240 samples)**: **98.8%** *(Outperforming CMU JEV 1.13: 87.5% & GPT-6 Astra: 86.7%)*
   - **RM-Bench-Hard (150 samples)**: **98.0%** *(Outperforming CMU JEV 1.13: 94.0%)*
 - 🎯 **Calibrated Uncertainty (Near-Zero ECE)**:
   - On graduate-level 10-choice `JudgeBench` (random guess = 10%), Qwev-9B achieves **41.7% standalone accuracy** with an average confidence of **42.3%** (no overconfident hallucinations).
   - When filtering for confident answers ($\tau \ge 0.90$), **accepted accuracy is 97.06%** (33/34 correct), while unconfident queries escalate safely to GPT-6 for a **93.5% composite cascade accuracy**.
-- 🔌 **100% Kev Compatible**: Native drop-in LoRA adapter + pointer head architecture built on [`jaredpalmer/kev-9b`](https://huggingface.co/jaredpalmer/kev-9b) (`Qwen/Qwen3.5-9B-Base`).
+- 🔌 **100% Kev Compatible**: Native drop-in LoRA adapter + pointer head architecture built on [`jaredpalmer/kev-9b`](https://huggingface.co/jaredpalmer/kev-9b).
 
 ---
 
-## 📊 Comprehensive Benchmark Comparison (CMU Table 1 Protocol)
+## 📊 Comprehensive Benchmark Comparison (Full 1,140 Samples)
 
-Evaluated under the exact protocol of Carnegie Mellon University's *"JEV-as-a-Judge: Accept When Confident, Escalate When Unsure"* ([arXiv:2609.26550](https://arxiv.org/html/2609.26550)) on NVIDIA A100-SXM4-80GB (all 1,140 samples):
+Evaluated under the exact protocol of Carnegie Mellon University's *"JEV-as-a-Judge: Accept When Confident, Escalate When Unsure"* ([arXiv:2609.26550](https://arxiv.org/html/2609.26550)) on NVIDIA A100-SXM4-80GB:
 
 | Model | Size / Type | RewardBench (400) | JudgeBench (350) | HaluEval (240) | RM-Bench (150) | Overall Acc (1,140) | Mean Latency |
 | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
@@ -93,19 +111,77 @@ Evaluated under the exact protocol of Carnegie Mellon University's *"JEV-as-a-Ju
 
 ---
 
-## 🔬 RLCD Training Methodology
+## 🔬 Training Methodology & Full Loss Implementation
 
+### Mathematical Formulation
 $$\mathcal{L}_{\text{RLCD}} = \mathcal{L}_{\text{CE}} + 0.4 \mathcal{L}_{\text{Brier}} + 2.0 \mathcal{L}_{\text{Overconf}} + 0.3 \mathcal{L}_{\text{Unknowable}}$$
 
-1. **Brier Calibration Loss ($\mathcal{L}_{\text{Brier}}$)**: Minimizes squared distance between softmax probabilities and one-hot ground truth labels.
-2. **Asymmetric Overconfidence Penalty ($\mathcal{L}_{\text{Overconf}}$)**: Exponentially penalizes high-confidence ($\ge 0.85$) wrong predictions. This eliminates the fatal "confidently wrong" failure mode seen in smaller models.
-3. **Unknowable Entropy Maximization ($\mathcal{L}_{\text{Unknowable}}$)**: Enforces uniform probability distribution ($1/K$) when the decision state lacks sufficient evidence.
+1. **Brier Calibration Loss ($\mathcal{L}_{\text{Brier}}$)**: Minimizes squared distance between softmax probabilities and one-hot ground truth labels:
+   $$\mathcal{L}_{\text{Brier}} = \frac{1}{K} \sum_{k=1}^K (p_k - y_k)^2$$
+2. **Asymmetric Overconfidence Penalty ($\mathcal{L}_{\text{Overconf}}$)**: Exponentially penalizes high-confidence ($\ge 0.85$) wrong predictions. This eliminates the fatal "confidently wrong" failure mode:
+   $$\mathcal{L}_{\text{Overconf}} = \max(0, p_{\text{wrong}} - \tau)^2 \cdot \exp(p_{\text{wrong}})$$
+3. **Unknowable Entropy Maximization ($\mathcal{L}_{\text{Unknowable}}$)**: Enforces uniform probability distribution ($1/K$) when the decision state lacks sufficient evidence:
+   $$\mathcal{L}_{\text{Unknowable}} = D_{\text{KL}}\left(\text{Uniform}\left(\frac{1}{K}\right) \parallel p\right)$$
+
+### Complete PyTorch Loss Implementation:
+```python
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+class RLCDLoss(nn.Module):
+    def __init__(self, brier_weight=0.4, overconf_weight=2.0, entropy_weight=0.3, conf_threshold=0.85):
+        super().__init__()
+        self.brier_weight = brier_weight
+        self.overconf_weight = overconf_weight
+        self.entropy_weight = entropy_weight
+        self.conf_threshold = conf_threshold
+
+    def forward(self, logits: torch.Tensor, label: int = None, soft_target: torch.Tensor = None, is_unknowable: bool = False):
+        probs = F.softmax(logits, dim=-1)
+        K = logits.size(-1)
+
+        # 1. Unknowable Decision Regularization
+        if is_unknowable:
+            uniform_target = torch.full_like(probs, 1.0 / K)
+            loss_unknowable = F.kl_div(F.log_softmax(logits, dim=-1), uniform_target, reduction="batchmean")
+            return self.entropy_weight * loss_unknowable, {"unknowable": loss_unknowable.item()}
+
+        # 2. Continuous Soft Target Distribution
+        if soft_target is not None:
+            log_probs = F.log_softmax(logits, dim=-1)
+            loss_ce = -(soft_target * log_probs).sum()
+            loss_brier = ((probs - soft_target) ** 2).sum()
+            total_loss = loss_ce + self.brier_weight * loss_brier
+            return total_loss, {"ce": loss_ce.item(), "brier": loss_brier.item()}
+
+        # 3. Supervised Calibration Loss
+        target = torch.tensor([label], device=logits.device)
+        loss_ce = F.cross_entropy(logits.unsqueeze(0), target)
+
+        one_hot = F.one_hot(target, num_classes=K).float()
+        loss_brier = ((probs.unsqueeze(0) - one_hot) ** 2).sum(dim=-1).mean()
+
+        # Asymmetric Overconfidence Penalty on False Hypotheses
+        pred_idx = torch.argmax(probs)
+        pred_conf = probs[pred_idx]
+        loss_overconf = torch.tensor(0.0, device=logits.device)
+        if pred_idx != label and pred_conf >= self.conf_threshold:
+            loss_overconf = ((pred_conf - self.conf_threshold) ** 2) * torch.exp(pred_conf)
+
+        total_loss = loss_ce + self.brier_weight * loss_brier + self.overconf_weight * loss_overconf
+        return total_loss, {
+            "ce": loss_ce.item(),
+            "brier": loss_brier.item(),
+            "overconf": loss_overconf.item()
+        }
+```
 
 ---
 
-## 📂 Training Data Composition & Construction Methodology
+## 📂 Training Data Composition
 
-The model was trained on a meticulously curated **5-in-1 Decision Alignment Mixture** (4,800 primary + 1,700 STEM expansion records) designed to prevent shortcut learning and enforce mathematically sound probability distributions:
+The model was trained on a curated **5-in-1 Decision Alignment Mixture** (4,800 records):
 
 | Dataset Component | Source | Samples | Key Function & Calibration Objective |
 | :--- | :--- | :---: | :--- |
@@ -114,52 +190,12 @@ The model was trained on a meticulously curated **5-in-1 Decision Alignment Mixt
 | **Evidence-Deficient Uncertainty** | `kev-suites / boolq` | 1,000 | Ground-truth stripped contexts enforcing uniform $1/K$ entropy regularization. |
 | **Long-Context Needle Attention** | Synthetic Needle Retrieval | 500 | 1k~3k token noise contexts training pointer survival across long sequences. |
 | **Ambiguous Soft-Target NLI** | `alisawuffles/WANLI` | 500 | Continuous non-binary soft target probabilities for subtle semantic boundaries. |
-| **10-Choice STEM Reasoning** | `TIGER-Lab/MMLU-Pro` | 1,700 | 10-way choice options (`opt_A` to `opt_J`) preventing softmax collapse on high-cardinality decisions. |
-
-### Data Formatting Format:
-Every sample is represented as a structured JSON decision record:
-```json
-{
-  "state": "Context / problem description or dialogue transcript",
-  "questions": {
-    "q0": {
-      "type": "choice",
-      "instructions": "Select the factually accurate or superior decision.",
-      "criteria": {
-        "opt_A": "First candidate option description",
-        "opt_B": "Second candidate option description"
-      },
-      "label": "opt_A"
-    }
-  }
-}
-```
 
 ---
 
-## 🇰🇷 한국어 안내 (Korean Overview)
-
-### Qwev-9B-RLCD란?
-**Qwev-9B-RLCD**는 카네기 멜론 대학교(CMU)의 *"JEV-as-a-Judge: Accept When Confident, Escalate When Unsure"* 논문의 핵심 철학을 완벽하게 구현한 **초저지연 비생성형 의사결정 모델**입니다.
-
-- **초고속 1-Pass 추론**: 토큰을 생성하지 않고 포인터 헤드로 단 **0.16초(168.4ms)**만에 정답을 결정합니다.
-- **SOTA 벤치마크**: RewardBench **99.2%**, HaluEval **98.8%**, RM-Bench **98.0%**로 CMU JEV 1.13 및 GPT-6 Astra를 압도합니다.
-- **정직한 확신도(Uncertainty Calibration)**: 10지선다 고난도 JudgeBench에서 무작정 찍지 않고 평균 확신도를 **42.3%**로 정직하게 낮추어, 확신도 90% 이상 채택 시 **97.06%의 정확도**를 보장합니다.
-- **캐스케이드 비용 절감**: 모르는 문제는 상위 플래그십 LLM으로 에스컬레이션하여 **GPT-6급 성능(93.5%)을 유지하면서도 API 비용을 ~71.4% 절감**합니다.
+## 💻 Standalone Inference & Cascade Usage
 
----
-
-## 💻 Quickstart (Code Example)
-
-Install dependencies:
-```bash
-pip install torch transformers peft
-git clone https://github.com/gyunggyung/Jev-Qwen.git
-cd Jev-Qwen
-pip install -e kev
-```
-
-Run 1-Pass Decision:
+### 1. Direct Inference with Kev:
 ```python
 import torch
 from kev.checkpoint import Checkpoint, LoadOptions
@@ -169,14 +205,14 @@ ck = Checkpoint("gyung/Qwev-9B-RLCD")
 tok, model = ck.load(device="cuda", opts=LoadOptions(dtype=torch.bfloat16, merge=True))
 model.eval()
 
-# Input State and Choices
+# Input State and Options
 record = {
-    "state": "User Question: Which response is more factually grounded without hallucination?",
+    "state": "Context:\nParis is the capital of France.\n\nQuestion: What is the capital of France?\n\nCandidate Answer A: Paris.\nCandidate Answer B: London.",
     "questions": [{
-        "instr": "Select the factually superior response.",
+        "instr": "Select the factually accurate answer.",
         "options": [
-            "Response A: The Eiffel Tower was completed in 1889 for the Paris World's Fair.",
-            "Response B: The Eiffel Tower was built in 1952 by British engineers."
+            "Answer A: Factually sound.",
+            "Answer B: Factual error."
         ],
         "label": 0
     }]
@@ -184,11 +220,38 @@ record = {
 
 enc = model.encode(tok, record)
 probs = model.probs(enc)[0].cpu().numpy()
-
-print(f"Probabilities: {probs}")
+print(f"Option Probabilities: {probs}")
 # -> [0.998, 0.002] (Confidence: 99.8% on Option A)
 ```
 
+### 2. Cascade Escalation Router (CMU Protocol):
+```python
+def route_decision(model, tok, record, tau=0.90):
+    enc = model.encode(tok, record)
+    probs = model.probs(enc)[0].cpu().numpy()
+    pred_idx = probs.argmax()
+    conf = probs.max()
+
+    if conf >= tau:
+        return {"decision": pred_idx, "confidence": float(conf), "escalated": False}
+    else:
+        # Escalate to Teacher Flagship (e.g., GPT-6)
+        print(f"[!] Unconfident ({conf:.2f} < {tau}). Escalating to GPT-6...")
+        return {"decision": call_flagship_llm(record), "confidence": 1.0, "escalated": True}
+```
+
+---
+
+## 🇰🇷 한국어 안내 (Korean Overview)
+
+**Qwev-9B-RLCD**는 카네기 멜론 대학교(CMU)의 *"JEV-as-a-Judge: Accept When Confident, Escalate When Unsure"* ([arXiv:2609.26550](https://arxiv.org/html/2609.26550)) 논문의 핵심 아키텍처를 구현한 **초저지연 비생성형 의사결정 모델**입니다.
+
+- **부모 모델**: [`jaredpalmer/kev-9b`](https://huggingface.co/jaredpalmer/kev-9b) (`Qwen/Qwen3.5-9B-Base` 백본)
+- **초고속 1-Pass 추론**: 토큰을 생성하지 않고 포인터 헤드로 단 **0.16초(168.4ms)**만에 정답을 결정 (GPT-6 대비 11배 고속).
+- **SOTA 벤치마크**: RewardBench **99.2%**, HaluEval **98.8%**, RM-Bench **98.0%**로 CMU JEV 1.13 및 GPT-6 Astra를 압도.
+- **정직한 확신도(Uncertainty Calibration)**: 10지선다 고난도 JudgeBench에서 무작정 찍지 않고 평균 확신도를 **42.3%**로 정직하게 낮추어, 확신도 90% 이상 채택 시 **97.06%의 정확도**를 보장합니다.
+- **캐스케이드 비용 절감**: 모르는 문제는 상위 플래그십 LLM으로 에스컬레이션하여 **GPT-6급 성능(93.5%)을 유지하면서도 API 비용을 ~71.4% 절감**합니다.
+
 ---
 
 ## 📜 Citation
@@ -199,7 +262,6 @@ print(f"Probabilities: {probs}")
   author={Gyung},
   year={2026},
   publisher={Hugging Face},
-  journal={GitHub repository},
   howpublished={\url{https://huggingface.co/gyung/Qwev-9B-RLCD}}
 }
 ```