Upload README.md with huggingface_hub

README.md

@@ -1,160 +1,77 @@
 ---
+license: apache-2.0
+library_name: sentence-transformers
+pipeline_tag: text-classification
+language:
+- en
 tags:
+- modernbert
 - ColBERT
 - multi-vector
 - PyLate
-- sentence-transformers
 - typed-decisions
 - zero-shot-classification
 - decision-making
-pipeline_tag: text-classification
-library_name: sentence-transformers
-license: apache-2.0
-language:
-- en
-base_model: lightonai/LateOn
-metrics:
-- accuracy
-- brier_score
-- log_loss
 ---
 
-<div align="center">
-
 # ⚡ Tasksource-JEV-Nano (`tasksource-jev-nano-v0`)
 
-**SOTA High-Throughput Native Decision Model (<200M Parameters)**  
+**SOTA High-Throughput Native Decision Model (\<200M Parameters)**
 *Decoupled Multi-Vector Late Interaction for Fast, Calibrated System-One Decisions*
 
-[![Decision Index](https://img.shields.io/badge/Decision%20Index%20Raw-30.65-blue)](https://github.com/sileod/decision-models)
-[![Decision Index Skill](https://img.shields.io/badge/Decision%20Index%20Skill-9.58-green)](https://github.com/sileod/decision-models)
-[![JevBench Composite](https://img.shields.io/badge/JevBench%20Composite-63.14%2F100-purple)](https://github.com/sileod/decision-models)
-[![Classifier Benchmark v2](https://img.shields.io/badge/Classifier--Benchmark%20v2-55.78%25-green)](https://github.com/sileod/decision-models)
-[![License](https://img.shields.io/badge/License-Apache%202.0-green.svg)](https://opensource.org/licenses/Apache-2.0)
-[![Parameters](https://img.shields.io/badge/Parameters-149M-orange.svg)](https://huggingface.co/answerdotai/ModernBERT-base)
-[![Context Length](https://img.shields.io/badge/Context-8192-purple.svg)](https://huggingface.co/answerdotai/ModernBERT-base)
-
-</div>
-
----
-
-### What is Tasksource-JEV-Nano?
+[![Decision Index](https://img.shields.io/badge/Decision_Index_Raw-32.03-blue)](https://github.com/sileod/decision-models) [![Decision Index Skill](https://img.shields.io/badge/Decision_Index_Skill-10.32-green)](https://github.com/sileod/decision-models)
 
-**Tasksource-JEV-Nano** is a compact (~149M parameter) decision model that picks the optimal action, category, or verdict from a candidate list using **token-level multi-vector late interaction** rather than conventional classification heads.
+Weights updated 2026-10-06 to the step-4000 single checkpoint (plain grouped multi-question exposure, 16k unique requests) — no soup. Previous weights scored Raw 30.65 / Skill 9.58; current weights score Raw 32.03 / Skill 10.32 on the same full official suite. It uses the simple release inference path: state + single question (`S + Q_i`) with independently encoded options.
 
-Built on [`answerdotai/ModernBERT-base`](https://huggingface.co/answerdotai/ModernBERT-base) and [`lightonai/LateOn`](https://huggingface.co/lightonai/LateOn), it handles all three fundamental System-One decision types:
-1. **Choice**: Selecting the best candidate among variable numbers of options ($K=2$ to $K=150+$).
-2. **NOUL**: Yes/no uncertainty decisions.
-3. **Score**: Quantitative ranking and graded scales.
-
----
-
-### Key Architectural Strengths
-
-- **Reusable State Caching**: The situation context (state + question) is encoded **once** into token vectors. Evaluating 1, 10, or 100 candidate options reuses this cached state without re-encoding the context, delivering up to **5.45x speedup**.
-- **Exact Candidate Permutation Equivariance**: Unlike encoder-concatenation classifiers where candidate order biases logits, late-interaction candidate scoring is mathematically independent across candidates. Permuting the option list guarantees an identically permuted output (max diff: 0.00e+00).
-- **Sub-40ms Low-Latency Inference**: Operates in ~30–37 ms per decision on standard GPUs, requiring only ~1.5 GB VRAM.
-
----
+Built on [`answerdotai/ModernBERT-base`](https://huggingface.co/answerdotai/ModernBERT-base) and [`lightonai/LateOn`](https://huggingface.co/lightonai/LateOn) (~149M params, 8192 context). Handles choice (K=2 to 150+), NOUL, and score decisions.
 
 ## 📊 Benchmark Evaluations
 
-All evaluations below are measured on official benchmark suites under zero-shot out-of-distribution conditions:
-
-### 🏆 Sub-200M Peer Comparison
-
-| Model | Size | Decision Index Raw | Classifier-Benchmark v2 | JevBench v1.3 | Banking77 ($K=77$) | CLINC150 ($K=151$) | Latency (p50) |
-| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
-| **Tasksource-JEV-Nano-v0** | **149M** | **30.65** 🏆 | **55.78%** 🏆 | **63.14** 🏆 | **44.22%** 🏆 | **67.04%** 🏆 | **37.6 ms** |
-| GLiClass Modern-Large | 340M | — | 53.80% | ~55.9 | 40.20% | ~58.4% | ~85 ms |
-| GLiNER2.5 Base | 200M | — | 54.20% | ~60.0 | 41.50% | ~61.0% | ~62 ms |
-| GLiClass Base | 150M | — | 50.70% | ~51.2 | 37.30% | ~54.1% | ~45 ms |
-| Laya Multilingual | 150M | — | 46.10% | ~51.7 | 1.80% *(collapses)* | ~38.2% | ~50 ms |
-| GLiClass Edge | 35M | — | 35.20% | ~35.3 | 24.10% | ~31.0% | ~18 ms |
-
----
-
 ### Official Decision Index 0.2.1 (Full 100% Suite: 150,759 Requests)
 
-Evaluated on the full official **Decision Index 0.2.1** test suite covering all 38 scored benchmarks with **100% suite coverage**:
-
-- **Balanced Raw Index**: **30.65 / 100**
-- **Balanced Skill Index (Chance-Corrected)**: **9.58 / 100**
-- **Breadth Skill Index**: **8.39 / 100**
-- **Suite Coverage**: **100.0%** (150,759 / 150,759 requests completed, 0 errors)
-
-#### Area Breakdown (Official 0.2.1 Weights):
-- **Retrieval & Intent Classification**: **40.37%** (Skill: **20.84%**)
-  - *CLINC150+OOS*: **67.04%** macro-F1 (151 classes, 5,500 requests)
-  - *BANKING77*: **40.79%** macro-F1 / **44.22%** accuracy (77 classes, 3,080 requests)
-- **Arts & Human Taste**: **33.73%** (Skill: **9.32%**)
-  - *BPoMP*: **55.54%** accuracy (5,000 requests)
-  - *Humicroedit*: **55.52%** accuracy (2,628 requests)
-  - *New Yorker Caption Matching*: **34.66%** accuracy
-- **Language Understanding**: **32.21%** (Skill: **4.34%**)
-  - *WinoGrande*: **51.85%** accuracy
-  - *ANLI*: **32.63%** macro-F1 (3,200 requests)
-  - *HellaSwag*: **30.69%** accuracy
-- **Tools & Automation**: **26.70%** (Skill: **15.18%**)
-  - *BFCL*: **62.60%** field accuracy / **34.95%** case-exact (1,694 requests)
-  - *API-Bank*: **31.30%** accuracy (53 API tools, 508 requests)
-  - *When2Call*: **30.70%** accuracy
-- **Knowledge & Reasoning**: **23.16%** (Skill: **2.23%**)
-  - *MMLU*: **29.72%** accuracy (all 14,033 questions across 57 subjects)
-  - *RAGTruth*: **47.8%** hallucination detection F1
+- **Balanced Raw Index**: **32.03 / 100**
+- **Balanced Skill Index (Chance-Corrected)**: **10.32 / 100**
+- **Breadth Skill Index**: 9.01 / 100
+- **Suite Coverage**: 100.0% (150,759 / 150,759 requests, 1 error)
 
----
+#### Area Breakdown (Official 0.2.1 Weights)
 
-## 🚀 Usage Guide
+- **Retrieval & Intent Classification**: 43.02% (Skill: 23.09%)
+- **Arts & Human Taste**: 36.90% (Skill: 13.36%)
+- **Language Understanding**: 35.07% (Skill: 5.22%)
+- **Tools & Automation**: 24.92% (Skill: 13.50%)
+- **Knowledge & Reasoning**: 23.63% (Skill: 2.12%)
 
-### Installation
+## 🚀 Usage
 
-```bash
-pip install -U pylate torch
+```python
+from modernjev.multivector.lateon import LateOnDecisionScorer
+
+scorer = LateOnDecisionScorer(model_name_or_path="tasksource/tasksource-jev-nano-v0")
+out = scorer.decide_uncached(
+    states=["Customer reports unauthorized international wire transfer of $4,500."],
+    questions=["Select the appropriate fraud mitigation routing:"],
+    options=[["approve and monitor silently", "challenge with a push notification",
+              "freeze the account and call the customer", "decline and file a report"]],
+    kinds=["choice"],
+)
+print(out.probabilities)
 ```
 
-### Python Inference with PyLate
+Or raw PyLate:
 
 ```python
 from pylate import models
-import torch
-
-# Load the trained Decision Index champion model
-model = models.ColBERT("tasksource/tasksource-jev-nano-v0")
-
-# 1. Provide the decision context (State + Question)
-state = "Customer reports unauthorized international wire transfer of $4,500."
-question = "Select the appropriate fraud mitigation routing:"
-context = [state + "\nQuestion: " + question]
-
-# 2. Provide candidate decision options
-options = [
-    "approve and monitor silently",
-    "challenge with a push notification",
-    "freeze the account and call the customer",
-    "decline and file a report",
-]
-
-# 3. State is encoded once; each option is scored via token MaxSim
-ctx = model.encode(context, is_query=False, convert_to_tensor=True)[0]
-opts = model.encode(options, is_query=True, convert_to_tensor=True)
-scores = torch.stack([(o @ ctx.T).max(dim=1).values.sum() for o in opts])
-
-# 4. Calibrated temperature softmax probabilities
-temperature = 0.3236  # Calibrated model temperature
-probs = torch.softmax(scores / temperature, dim=0)
-
-for opt, p in zip(options, probs):
-    print(f"  {opt:45s} -> {p:.4f}")
+model = models.ColBERT(model_name_or_path="tasksource/tasksource-jev-nano-v0")
+ctx = model.encode(["<state>\nQuestion: <question>"], is_query=False, convert_to_tensor=True)[0]
+opts = model.encode(["<option 1>", "<option 2>"], is_query=True, convert_to_tensor=True)
 ```
 
----
-
 ## 📜 Citation
 
 ```bibtex
-@misc{sileo2026jevnanov0,
-  title={Tasksource-JEV-Nano-v0: Decoupled Multi-Vector Late Interaction for Typed Decisions},
+@misc{sileo2026jevnano,
+  title={Tasksource-JEV-Nano: Decoupled Multi-Vector Late Interaction for Typed Decisions},
   author={Sileo, Damien},
   year={2026},
   howpublished={\url{https://huggingface.co/tasksource/tasksource-jev-nano-v0}},