Update README.md

README.md

@@ -1,117 +1,969 @@
 ---
 license: apache-2.0
-language: [en]
+language:
+- en
 library_name: transformers
-tags: [decision-model, system-one, calibrated-decisions, multiple-choice, zero-shot-classification, falcondec]
+base_model: jhu-clsp/ettin-encoder-150m
+pipeline_tag: zero-shot-classification
+tags:
+- decision-model
+- system-one
+- falcondec
+- lightdec
+- calibrated-decisions
+- multiple-choice
+- intent-classification
+- customer-support
+- natural-language-inference
+- code
+- guardrails
+- agents
+- selective-prediction
+- falconsai
 ---
-# FalconDec v1.0.0 — Falcon Decision
 
-Single-pass, typed (`choice` / `noul` / `score`), calibrated closed-set decision model. Successor to
-[`Falconsai/proof_v2`](https://huggingface.co/Falconsai/proof_v2).
+# Falconsai/LightDec
 
-* Built with FalconDec notebook V2 · backbone `jhu-clsp/ettin-encoder-150m` · mode `scratch` · preset `standard` · lineage: jhu-clsp/ettin-encoder-150m
-* Parameters: 159.7M · weights: fp16 305 MB, int8 153 MB
-* Layout: `[CLS] question [SEP] [MASK] opt1 … [MASK] optk [SEP] state [SEP]`; set-transformer option head;
-  temperature per (question type × option-count bucket).
+**[View in Model Surgeon](https://surgeon.falcons.ai/?hub=Falconsai/LightDec)**
 
-## Usage
+**A lightweight, single-pass, typed, calibrated decision model for agentic systems.** Give it a **state** (text, code or JSON), one or more **typed questions** (`choice`, `noul` yes/no, `score` ordinal) and a closed set of options. It returns a calibrated probability for every option, from one encoder pass per question.
+
+LightDec is the FalconDec architecture trained with **FalconDec notebook V2** on the **`standard`** data preset. That run adds agent-specific decisions (AgentTrek, Counsel, HotpotQA) to a balanced mix of 58 test tasks across 9 domains. This page is both the **model card** and the **developer guide**: how to load it, test it, and use it as a decision component in an agentic system.
+
+> **At a glance.** Test accuracy **0.725** (micro and task-macro) on 17,498 decisions from 58 tasks, with ECE **0.025**. At a 0.70 confidence threshold, LightDec answers **56%** of decisions at **89.6%** accuracy and defers the rest. Weights: **319 MB** fp16, **161 MB** int8. It is strongest on support routing, code understanding, intents, guardrails and agent-step checks, and weakest on multi-step arithmetic, date and table reasoning, and very wide label sets. Evaluate it on your own traffic (§6.4) before acting on its answers.
+
+## Contents
+
+1. [Model summary](#1-model-summary)
+2. [Results](#2-results)
+3. [Intended and out-of-scope uses](#3-intended-and-out-of-scope-uses)
+4. [The Hub repository](#4-the-hub-repository)
+5. [Install, load and read a result](#5-install-load-and-read-a-result)
+6. [Testing the model](#6-testing-the-model)
+7. [Using it in an agentic system](#7-using-it-in-an-agentic-system)
+8. [Tuning for your domain](#8-tuning-for-your-domain)
+9. [Architecture](#9-architecture)
+10. [Training data](#10-training-data)
+11. [Training procedure and calibration](#11-training-procedure-and-calibration)
+12. [Operational notes](#12-operational-notes)
+13. [Bias, risks and limitations](#13-bias-risks-and-limitations)
+14. [Versioning and lineage](#14-versioning-and-lineage)
+15. [API reference](#15-api-reference)
+16. [Citation and references](#16-citation-and-references)
+
+---
+
+## 1. Model summary
+
+| | |
+|---|---|
+| **Model** | LightDec: the FalconDec architecture, notebook V2, `standard` preset. The checkpoint's own config reports `FalconDec` version `1.0.0` |
+| **Task** | Closed-set decisions: given a state, a question and 2–N options, return a calibrated probability per option |
+| **Question types** | `choice` (pick one), `noul` (yes/no, returns P(true)), `score` (ordinal rubric, returns the expected level) |
+| **Backbone** | [`jhu-clsp/ettin-encoder-150m`](https://huggingface.co/jhu-clsp/ettin-encoder-150m) (ModernBERT-style encoder), fully fine-tuned |
+| **Decision head** | Option-marker scoring plus a permutation-equivariant set-transformer head (§9) |
+| **Parameters** | ≈160M (Hub reports 0.2B) |
+| **Weights** | fp16 **319 MB** (`model.safetensors`) · per-channel int8 **161 MB** (`compact-int8/model_int8.safetensors`) |
+| **Context** | 512 tokens; automatically 2,048 for questions with more than 24 options; a tournament above 96 options |
+| **Calibration** | One temperature per (question type × option-count bucket), stored in the checkpoint and applied automatically |
+| **Inference cost** | One encoder pass per question. The same architecture (`Falconsai/proof_v3`) measured 15.7 ms p50 for one question on a GPU; LightDec's own latency is not in its report (measure with §6.3) |
+| **Output** | Probabilities, the chosen option, confidence, a `defer` flag, `p_true` (noul) and `expected_level` (score) |
+| **Language** | English, plus code in Python, Java, JavaScript, PHP, Ruby, Go and C |
+| **Custom code** | `falcondec_modeling.py` ships with the weights and holds the model and all inference logic. Load it with `importlib` (§5); `AutoModel.from_pretrained` alone won't build the decision head |
+| **License** | Apache-2.0 for the weights and code. Check each training dataset's license before redistributing derived data |
+
+The model has no generative component. It can only rank the options you give it, so it cannot produce text outside that set.
+
+---
+
+## 2. Results
+
+All numbers come from this checkpoint's `falcondec_report.json`: one run, `standard` preset, 2 epochs, seed 42, `MODE="scratch"`, notebook V2.
+
+### 2.1 At a glance
+
+| Metric | Value |
+|---|---|
+| Test decisions / tasks | 17,498 / 58 (10 of them held out) |
+| Test accuracy, micro | **0.725** |
+| Test accuracy, task-macro | **0.725** |
+| Held-out tasks, task-macro (10 tasks never trained on) | **0.567** |
+| Expected calibration error (ECE, 15 bins) | **0.025** |
+| Negative log-likelihood / Brier score | 0.652 / 0.358 |
+| Area under the risk–coverage curve (AURC, lower is better) | **0.097** |
+| Ordinal (`score`) mean absolute error, in levels | 0.572 |
+| Coverage / accuracy at confidence ≥ 0.70 | **56.4% / 0.896** |
+| Weights | fp16 319 MB · int8 161 MB |
+
+**Selective prediction is the headline.** Calibration is good (ECE 0.025), so the confidence score is a reliable gate. Acting only on decisions with confidence ≥ 0.70 covers 56% of traffic at 89.6% accuracy, against 72.5% accuracy when answering everything. That is the property an agent loop needs: answer the easy majority locally, and hand the rest to an LLM or a human.
+
+### 2.2 Per domain
+
+| Domain | Test decisions | Task-macro accuracy |
+|---|---|---|
+| support | 600 | **0.997** |
+| code | 2,350 | **0.864** |
+| intents | 1,200 | **0.773** |
+| guardrails | 1,094 | **0.768** |
+| agentic | 717 | **0.754** |
+| workflows | 2,000 | **0.689** |
+| policy | 2,700 | **0.669** |
+| reasoning | 5,637 | **0.667** |
+| classification | 1,200 | **0.584** |
+
+### 2.3 Per task
+
+*Held-out tasks were never used for training, calibration or model selection. "proof_v2 (card)" lists proof_v2's published score for the same source and task (different samples; indicative only).*
+
+| Domain | Task | n | Chance | **Accuracy** | ECE | proof_v2 (card) |
+|---|---|---|---|---|---|---|
+| support | `bitext/route` | 300 | 0.200 | **1.000** | 0.002 | 0.958 |
+| support | `bitext/category` | 300 | 0.172 | **0.993** | 0.008 |  |
+| code | `codexglue/lang_id` | 300 | 0.235 | **1.000** | 0.004 | 0.997 |
+| code | `mbpp/solution` | 300 | 0.250 | **0.987** | 0.014 | 0.992 |
+| code | `codexglue/code_to_doc` | 300 | 0.256 | **0.977** | 0.016 | 0.969 |
+| code | `codexglue/doc_to_code` | 300 | 0.274 | **0.973** | 0.022 | 0.961 |
+| code | `codexglue/func_name` | 275 | 0.263 | **0.938** | 0.021 | 0.901 |
+| code | `bigclonebench/clone` | 300 | 0.500 | **0.863** | 0.098 | 0.383 |
+| code | `humaneval/completion` *(held out)* | 119 | 0.394 | **0.756** | 0.152 | 0.575 |
+| code | `mbpp/bugspot` | 156 | 0.413 | **0.731** | 0.086 | 0.475 |
+| code | `devign/vulnerability` | 300 | 0.500 | **0.553** | 0.026 | 0.542 |
+| intents | `banking77/intent` *(held out)* | 300 | 0.317 | **0.923** | 0.037 | 0.883 |
+| intents | `massive_en/intent` | 300 | 0.122 | **0.907** | 0.046 |  |
+| intents | `clinc150/intent` | 300 | 0.122 | **0.793** | 0.081 | 0.850 |
+| intents | `banking77/intent_77` *(held out)* | 300 | 0.013 | **0.470** | 0.200 |  |
+| guardrails | `jailbreak/detect` | 262 | 0.500 | **0.966** | 0.018 |  |
+| guardrails | `civil_comments/toxic` | 300 | 0.500 | **0.807** | 0.051 |  |
+| guardrails | `agentharm/refuse` *(held out)* | 416 | 0.500 | **0.654** | 0.178 |  |
+| guardrails | `prompt_injections/detect` *(held out)* | 116 | 0.500 | **0.647** | 0.272 |  |
+| agentic | `hotpotqa/retrieve` | 298 | 0.168 | **0.842** | 0.078 |  |
+| agentic | `hotpotqa/comparison_yes_no` | 17 | 0.500 | **0.824** | 0.185 |  |
+| agentic | `counsel/step_has_error` | 201 | 0.500 | **0.791** | 0.093 |  |
+| agentic | `counsel/critique_quality` | 201 | 0.333 | **0.557** | 0.094 |  |
+| workflows | `typed_decisions/customer_service` | 500 | 0.280 | **0.720** | 0.099 |  |
+| workflows | `typed_decisions/security_incidents` | 500 | 0.340 | **0.712** | 0.140 |  |
+| workflows | `typed_decisions/agent_trace_observability` | 500 | 0.300 | **0.696** | 0.106 |  |
+| workflows | `typed_decisions/invoice_processing` | 500 | 0.350 | **0.628** | 0.101 |  |
+| policy | `policy/access_control_transfer` | 300 | 0.333 | **1.000** | 0.001 |  |
+| policy | `policy/return_window_transfer` | 300 | 0.333 | **1.000** | 0.019 |  |
+| policy | `policy/free_shipping_transfer` | 300 | 0.500 | **0.850** | 0.067 |  |
+| policy | `policy/sla_urgency_transfer` | 300 | 0.250 | **0.713** | 0.214 |  |
+| policy | `policy/refund_approval_transfer` | 300 | 0.333 | **0.710** | 0.219 |  |
+| policy | `policy/count_threshold_transfer` | 300 | 0.179 | **0.523** | 0.085 |  |
+| policy | `policy/invoice_total_transfer` | 300 | 0.500 | **0.523** | 0.020 |  |
+| policy | `policy/invoice_overdue_transfer` | 300 | 0.500 | **0.407** | 0.364 |  |
+| policy | `policy/table_extreme_transfer` | 300 | 0.240 | **0.290** | 0.036 |  |
+| reasoning | `qasc/mcq` | 300 | 0.125 | **0.983** | 0.007 |  |
+| reasoning | `snli/must_be_true` | 300 | 0.333 | **0.970** | 0.029 | 0.908 |
+| reasoning | `snli/contradicts` | 300 | 0.333 | **0.967** | 0.035 | 0.892 |
+| reasoning | `scitail/support` | 300 | 0.500 | **0.957** | 0.030 |  |
+| reasoning | `sciq/mcq` | 300 | 0.250 | **0.950** | 0.023 | 0.692 |
+| reasoning | `snli/nli` | 594 | 0.333 | **0.837** | 0.035 |  |
+| reasoning | `mnli/claim` | 300 | 0.333 | **0.807** | 0.072 | 0.492 |
+| reasoning | `boolq/yes_no` | 300 | 0.500 | **0.783** | 0.071 | 0.717 |
+| reasoning | `gsm8k/math` | 300 | 0.250 | **0.637** | 0.050 | 0.275 |
+| reasoning | `commonsense_qa/mcq` | 296 | 0.200 | **0.611** | 0.058 | 0.442 |
+| reasoning | `arc_easy/mcq` *(held out)* | 300 | 0.250 | **0.553** | 0.057 | 0.425 |
+| reasoning | `openbookqa/mcq` | 300 | 0.250 | **0.550** | 0.068 | 0.292 |
+| reasoning | `winogrande/blank` | 300 | 0.500 | **0.540** | 0.089 |  |
+| reasoning | `arc_challenge/mcq` *(held out)* | 300 | 0.250 | **0.423** | 0.085 | 0.308 |
+| reasoning | `anli/nli` | 300 | 0.333 | **0.393** | 0.185 |  |
+| reasoning | `mmlu/mcq` *(held out)* | 300 | 0.250 | **0.393** | 0.094 |  |
+| reasoning | `hellaswag/continuation` | 300 | 0.250 | **0.383** | 0.171 |  |
+| reasoning | `aqua_rat/math` | 247 | 0.200 | **0.259** | 0.071 |  |
+| classification | `ag_news/topic` | 300 | 0.250 | **0.847** | 0.051 |  |
+| classification | `yelp/score` | 300 | 0.200 | **0.640** | 0.069 |  |
+| classification | `emotion/6way` *(held out)* | 300 | 0.167 | **0.480** | 0.049 |  |
+| classification | `sst5/score` *(held out)* | 300 | 0.200 | **0.370** | 0.086 |  |
+
+### 2.4 Comparison with proof_v2 (indicative)
+
+On the 22 tasks that both this report and proof_v2's model card cover, LightDec's task-macro accuracy is **0.807 vs 0.679**, and it scores higher on **20 of 22**. The largest gains are on the tasks proof_v2 reported as weak:
+
+| Task | proof_v2 (card) | LightDec |
+|---|---|---|
+| BigCloneBench clone detection | 0.383 | **0.863** |
+| GSM8K (4-option numeric) | 0.275 | **0.637** |
+| MultiNLI claim | 0.492 | **0.807** |
+| OpenBookQA | 0.292 | **0.550** |
+| MBPP bug spotting | 0.475 | **0.731** |
+| HumanEval completion *(held out)* | 0.575 | **0.756** |
+| ARC-Challenge *(held out)* | 0.308 | **0.423** |
+
+LightDec is lower on CLINC150 (0.793 vs 0.850) and MBPP task→solution (0.987 vs 0.992).
+
+These are **different test samples and, for some tasks, different question formats**. For example, LightDec's MultiNLI task is three-way NLI, and its bug-spotting mutants are verified to fail the unit tests. The like-for-like head-to-head, which runs proof_v2 on identical decisions (notebook cell 23), **did not run** for this checkpoint (`head_to_head: null`).
+
+### 2.5 Comparison with Laya and TypeSafe Jev (indicative)
+
+| Benchmark | LightDec | Laya | TypeSafe Jev 1.13.0 |
+|---|---|---|---|
+| typed-decisions test (2,000 decisions, 4 workflows) | 0.689 | 0.766 (fine-tuned) · 0.362 (zero-shot) | 0.727 |
+| AG News (4 labels) | 0.847 | 0.950 | 0.910 |
+| DAIR Emotion, 6 labels *(held out for LightDec)* | 0.480 | 0.595 | 0.480 |
+| Banking77, all 77 labels in one question *(held out)* | 0.470 | 0.425 | 0.870 (72 labels) |
+| SST-5 (ordinal) *(held out)* | 0.370 | 0.372 | — |
+
+Laya's numbers are from its own benchmark report; Jev's are third-party published. LightDec was trained on the typed-decisions training split, like the fine-tuned Laya checkpoint. **On typed-decisions LightDec trails both** (the teacher-agreement ceiling is 0.735 and the majority-class baseline 0.461). It matches Jev on Emotion, edges Laya on all-77 Banking77, and trails both on AG News. LightDec is 2.6× smaller than Laya's 421M English checkpoint.
+
+---
+
+## 3. Intended and out-of-scope uses
+
+### Intended
+
+| Use | Measured evidence |
+|---|---|
+| **Support and ticket routing** | Bitext route 1.000, Bitext category 0.993; Banking77 (held out, 2–5 options) 0.923; MASSIVE 0.907 |
+| **Code understanding against a menu** | Language ID 1.000; code↔description 0.973–0.977; task→solution 0.987; function naming 0.938 |
+| **Guardrails** | Jailbreak detection 0.966; toxicity 0.807. Held out: prompt-injection 0.647, AgentHarm refusal 0.654, so recalibrate and test on your own traffic |
+| **Agent loops** | Retrieval routing (HotpotQA) 0.842; "does this agent step contain an error?" (Counsel) 0.791 |
+| **Statement verification** | SNLI must-be-true / contradicts 0.970 / 0.967; SciTail 0.957; SciQ 0.950 |
+| **Selective automation** | 89.6% accuracy on the 56% of decisions with confidence ≥ 0.70 |
+
+### Out of scope
+
+- **Multi-step arithmetic and quantitative reasoning.** AQuA 0.259 (chance 0.20); counting and summing policies 0.52; comparing values in a table 0.290. GSM8K reaches 0.637 only because it is posed as 4-option multiple choice with near-miss distractors. Route real math to an LLM or code.
+- **Date reasoning in unfamiliar formats.** The invoice-overdue transfer test (ISO dates, whereas training used "Month DD, YYYY") scores **0.407, below chance, with ECE 0.364**. It is confidently wrong there. Normalise dates before asking, or compute them in code.
+- **Very wide label sets.** All 77 Banking77 intents in one question score 0.470. Pre-filter to a shortlist of about 20 options (§7.1).
+- **Hard commonsense and exam knowledge**: HellaSwag 0.383, ANLI 0.393, MMLU 0.393.
+- **Code security and correctness gating**: Devign 0.553 is near chance. Don't use it to approve code.
+- **Open-ended questions**: it always picks one of your options. Add "None of the above" when appropriate; it was trained with that option.
+- **Non-English text**, and **high-stakes decisions without human oversight**.
+
+---
+
+## 4. The Hub repository
+
+| File | Contents |
+|---|---|
+| `model.safetensors` | fp16 weights (319 MB): encoder, decision head and the temperature buffer |
+| `falcondec_config.json` | Layout (sequence lengths, option budgets), special-token ids, temperatures, defer threshold, version, lineage |
+| `encoder/` | Backbone configuration (the encoder is rebuilt from this, then the weights are loaded) |
+| `tokenizer/` | Tokenizer files |
+| `falcondec_modeling.py` | `FalconDec`, `load_falcondec`, `decide`, `score_items`, `save_falcondec` and the int8 codec |
+| `falcondec_report.json` | Training configuration, data counts, history, temperatures and all test results |
+| `compact-int8/` | The same model with per-channel int8 weights (`model_int8.safetensors`, 161 MB); a complete, self-contained directory with its own config, tokenizer and modeling file |
+| `README.md` | This card |
+
+**Pin a revision in production.** The repo can change, so pass a commit hash when you load.
+
+---
+
+## 5. Install, load and read a result
+
+```bash
+pip install torch "transformers>=4.48" safetensors huggingface_hub numpy
+```
+
+### 5.1 Load
+
+Save this helper as `lightdec.py` next to your code. Every example below uses it.
 
 ```python
-import importlib.util
+# lightdec.py
+import importlib.util, json, shutil
+from pathlib import Path
 from huggingface_hub import snapshot_download
-path = snapshot_download("<repo>")            # or a local FalconDec-v1.0.0 directory
-spec = importlib.util.spec_from_file_location("falcondec_modeling", f"{path}/falcondec_modeling.py")
-fdm = importlib.util.module_from_spec(spec); spec.loader.exec_module(fdm)
-model, tok = fdm.load_falcondec(path)          # add "/compact-int8" for the int8 artefact
-out = fdm.decide(model, tok, state={"body": "I was charged twice, refund me or I cancel."}, questions={
-    "dept": {"type": "choice", "instructions": "Which team?", "criteria": {"billing": "payments", "tech": "bugs"}},
-    "churn": {"type": "noul", "instructions": "Does the user threaten to leave?"},
-    "urgency": {"type": "score", "instructions": "How urgent?", "criteria": ["low", "medium", "high"]}})
-```
-
-## Test results
-
-Overall: micro acc **0.725**, task-macro acc **0.725**, ECE **0.025**,
-NLL 0.652, Brier 0.358, AURC 0.097.
-
-| task | domain | heldout | n | chance | acc | proof_v2 (card) | ece |
-|---|---|---|---|---|---|---|---|
-| counsel/critique_quality | agentic | False | 201 | 0.333 | 0.557 |  | 0.094 |
-| counsel/step_has_error | agentic | False | 201 | 0.500 | 0.791 |  | 0.093 |
-| hotpotqa/comparison_yes_no | agentic | False | 17 | 0.500 | 0.824 |  | 0.185 |
-| hotpotqa/retrieve | agentic | False | 298 | 0.168 | 0.842 |  | 0.078 |
-| sst5/score | classification | True | 300 | 0.200 | 0.370 |  | 0.086 |
-| emotion/6way | classification | True | 300 | 0.167 | 0.480 |  | 0.049 |
-| yelp/score | classification | False | 300 | 0.200 | 0.640 |  | 0.069 |
-| ag_news/topic | classification | False | 300 | 0.250 | 0.847 |  | 0.051 |
-| devign/vulnerability | code | False | 300 | 0.500 | 0.553 | 0.542 | 0.026 |
-| mbpp/bugspot | code | False | 156 | 0.413 | 0.731 | 0.475 | 0.086 |
-| humaneval/completion | code | True | 119 | 0.394 | 0.756 | 0.575 | 0.152 |
-| bigclonebench/clone | code | False | 300 | 0.500 | 0.863 | 0.383 | 0.098 |
-| codexglue/func_name | code | False | 275 | 0.263 | 0.938 | 0.901 | 0.021 |
-| codexglue/doc_to_code | code | False | 300 | 0.274 | 0.973 | 0.961 | 0.022 |
-| codexglue/code_to_doc | code | False | 300 | 0.256 | 0.977 | 0.969 | 0.016 |
-| mbpp/solution | code | False | 300 | 0.250 | 0.987 | 0.992 | 0.014 |
-| codexglue/lang_id | code | False | 300 | 0.235 | 1.000 | 0.997 | 0.004 |
-| prompt_injections/detect | guardrails | True | 116 | 0.500 | 0.647 |  | 0.272 |
-| agentharm/refuse | guardrails | True | 416 | 0.500 | 0.654 |  | 0.178 |
-| civil_comments/toxic | guardrails | False | 300 | 0.500 | 0.807 |  | 0.051 |
-| jailbreak/detect | guardrails | False | 262 | 0.500 | 0.966 |  | 0.018 |
-| banking77/intent_77 | intents | True | 300 | 0.013 | 0.470 |  | 0.200 |
-| clinc150/intent | intents | False | 300 | 0.122 | 0.793 | 0.850 | 0.081 |
-| massive_en/intent | intents | False | 300 | 0.122 | 0.907 |  | 0.046 |
-| banking77/intent | intents | True | 300 | 0.317 | 0.923 | 0.883 | 0.037 |
-| policy/table_extreme_transfer | policy | False | 300 | 0.240 | 0.290 |  | 0.036 |
-| policy/invoice_overdue_transfer | policy | False | 300 | 0.500 | 0.407 |  | 0.364 |
-| policy/count_threshold_transfer | policy | False | 300 | 0.179 | 0.523 |  | 0.085 |
-| policy/invoice_total_transfer | policy | False | 300 | 0.500 | 0.523 |  | 0.020 |
-| policy/refund_approval_transfer | policy | False | 300 | 0.333 | 0.710 |  | 0.219 |
-| policy/sla_urgency_transfer | policy | False | 300 | 0.250 | 0.713 |  | 0.214 |
-| policy/free_shipping_transfer | policy | False | 300 | 0.500 | 0.850 |  | 0.067 |
-| policy/access_control_transfer | policy | False | 300 | 0.333 | 1.000 |  | 0.001 |
-| policy/return_window_transfer | policy | False | 300 | 0.333 | 1.000 |  | 0.019 |
-| aqua_rat/math | reasoning | False | 247 | 0.200 | 0.259 |  | 0.071 |
-| hellaswag/continuation | reasoning | False | 300 | 0.250 | 0.383 |  | 0.171 |
-| anli/nli | reasoning | False | 300 | 0.333 | 0.393 |  | 0.185 |
-| mmlu/mcq | reasoning | True | 300 | 0.250 | 0.393 |  | 0.094 |
-| arc_challenge/mcq | reasoning | True | 300 | 0.250 | 0.423 | 0.308 | 0.085 |
-| winogrande/blank | reasoning | False | 300 | 0.500 | 0.540 |  | 0.089 |
-| openbookqa/mcq | reasoning | False | 300 | 0.250 | 0.550 | 0.292 | 0.068 |
-| arc_easy/mcq | reasoning | True | 300 | 0.250 | 0.553 | 0.425 | 0.057 |
-| commonsense_qa/mcq | reasoning | False | 296 | 0.200 | 0.611 | 0.442 | 0.058 |
-| gsm8k/math | reasoning | False | 300 | 0.250 | 0.637 | 0.275 | 0.050 |
-| boolq/yes_no | reasoning | False | 300 | 0.500 | 0.783 | 0.717 | 0.071 |
-| mnli/claim | reasoning | False | 300 | 0.333 | 0.807 | 0.492 | 0.072 |
-| snli/nli | reasoning | False | 594 | 0.333 | 0.837 |  | 0.035 |
-| sciq/mcq | reasoning | False | 300 | 0.250 | 0.950 | 0.692 | 0.023 |
-| scitail/support | reasoning | False | 300 | 0.500 | 0.957 |  | 0.030 |
-| snli/contradicts | reasoning | False | 300 | 0.333 | 0.967 | 0.892 | 0.035 |
-| snli/must_be_true | reasoning | False | 300 | 0.333 | 0.970 | 0.908 | 0.029 |
-| qasc/mcq | reasoning | False | 300 | 0.125 | 0.983 |  | 0.007 |
-| bitext/category | support | False | 300 | 0.172 | 0.993 |  | 0.008 |
-| bitext/route | support | False | 300 | 0.200 | 1.000 | 0.958 | 0.002 |
-| typed_decisions/invoice_processing | workflows | False | 500 | 0.350 | 0.628 |  | 0.101 |
-| typed_decisions/agent_trace_observability | workflows | False | 500 | 0.300 | 0.696 |  | 0.106 |
-| typed_decisions/security_incidents | workflows | False | 500 | 0.340 | 0.712 |  | 0.140 |
-| typed_decisions/customer_service | workflows | False | 500 | 0.280 | 0.720 |  | 0.099 |
-
-
-## Training
-
-Strictly proper objective (log + 0.5·spherical + 1.0·RPS for ordinal), soft teacher targets where available,
-task-balanced sampling (α = 0.5), NOTA augmentation (8%), per-epoch option reshuffling,
-AdamW + LLRD 0.9, EMA 0.999, 2 epoch(s), RLCD stage off/reverted.
-Training time 46.1 min on NVIDIA GeForce RTX 5090 Laptop GPU.
-
-## Limitations
-
-* Closed world: the model ranks the options you give it. Add "None of the above" when that is a valid answer (it was trained with it).
-* Multi-step arithmetic (GSM8K / AQuA) is a System-2 task; route low-confidence answers to an LLM (`defer` flag, threshold 0.7).
-* English only; code judgement (vulnerabilities, clones) is still weak; check calibration on your own traffic.
-
-## References
-
-Warner et al. 2024 (ModernBERT) · Weller et al. 2025 (Ettin) · Gneiting & Raftery 2007 (strictly proper scoring rules) ·
-Guo et al. 2017 (temperature scaling) · Geifman & El-Yaniv 2017 (selective classification) · Shao et al. 2024 (GRPO) ·
-Williams 1992 (REINFORCE) · Zaheer et al. 2017 / Lee et al. 2019 (Deep Sets / Set Transformer) · Hinton et al. 2015 (distillation) ·
-Laya (convaiinnovations) · TypeSafe Jev · Together Tev1.
+
+
+def load_lightdec(repo="Falconsai/LightDec", revision=None, variant="fp16", device=None, dtype=None):
+    """Returns (fdm, model, tokenizer). variant: "fp16" (319 MB) or "int8" (161 MB, dequantised on load)."""
+    path = Path(repo) if Path(repo).exists() else Path(snapshot_download(repo, revision=revision))
+    if variant == "int8":
+        path = path / "compact-int8"
+    fc = json.loads((path / "falcondec_config.json").read_text(encoding="utf-8"))
+    expected = fc.get("weights", "model.safetensors")
+    if not (path / expected).exists():                  # e.g. a renamed weight file in a processed copy
+        cands = sorted(path.glob("*.safetensors"))
+        if not cands:
+            raise FileNotFoundError(f"no .safetensors weights in {path}")
+        local = Path("lightdec_local") / variant
+        shutil.copytree(path, local, dirs_exist_ok=True)
+        shutil.copy(cands[0], local / expected)
+        path = local
+    spec = importlib.util.spec_from_file_location("falcondec_modeling", str(path / "falcondec_modeling.py"))
+    fdm = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(fdm)
+    model, tok = fdm.load_falcondec(str(path), device=device, dtype=dtype)   # cuda if available, else cpu
+    return fdm, model, tok
+```
+
+```python
+from lightdec import load_lightdec
+
+fdm, model, tok = load_lightdec()                      # or load_lightdec(revision="<commit>", variant="int8")
+print(model.fcfg["name"], model.fcfg["version"], round(model.num_parameters() / 1e6, 1), "M params")
+```
+
+If loading prints `[FalconDec] load warning: missing=… unexpected=…`, the weights didn't match the architecture. Treat that as a failed load (§6.1 checks for it).
+
+### 5.2 Decide
+
+`decide()` takes one state and any number of typed questions: either a list, or a Jev/Laya-style dict keyed by name.
+
+```python
+state = {"from": "user@acme.com", "subject": "Duplicate charge on invoice #4411",
+         "body": "We were billed twice for March. Please refund the duplicate today or we will cancel our plan."}
+
+out = fdm.decide(model, tok, state, {
+    "department": {"type": "choice", "instructions": "Which department should handle this request?",
+                   "criteria": {"billing": "invoices, payments, refunds", "technical": "bugs, outages",
+                                "sales": "pricing, contracts", "other": "everything else"}},
+    "urgency":    {"type": "score", "instructions": "How urgent is this request?",
+                   "criteria": ["not urgent", "soon", "critical deadline or blocking issue"]},
+    "churn_risk": {"type": "noul", "instructions": "Does the user threaten to cancel or leave?"},
+})
+a = out["answers"]
+print(a["department"]["choice"], round(a["department"]["confidence"], 3), a["department"]["defer"])
+print("urgency level", round(a["urgency"]["expected_level"], 2), "of", 2)
+print("P(churn)", round(a["churn_risk"]["p_true"], 3))
+```
+
+Plain options work too: `{"question": "Which team?", "options": ["Accounts", "Billing", "Shipping"]}`.
+
+### 5.3 Reading a result
+
+Each item in `out["results"]` (and `out["answers"][key]`) contains:
+
+| Field | Meaning |
+|---|---|
+| `key` | The question's name (dict input) or `None` |
+| `type` | `choice`, `noul` or `score` |
+| `choice` | The chosen key: the criteria key, the option text, `True`/`False` for `noul`, or the level index for `score` |
+| `choice_text` | The option text the model saw |
+| `confidence` | Calibrated probability of `choice` |
+| `probs` | The full distribution, keyed by `str(key)` |
+| `defer` | `True` when `confidence` is below the defer threshold (default 0.70, stored in the config): don't act on it (§7.3) |
+| `p_true` | `noul` only: calibrated P(yes) |
+| `expected_level` | `score` only: probability-weighted level (0 … k−1); better than the argmax for ordinal rubrics (test MAE 0.57 levels) |
+
+How to read them:
+
+- **Low confidence, spread probabilities**: the state doesn't support any option clearly. Defer, or add "None of the above".
+- **`noul` near 0.5**: genuinely ambiguous. Ask for more information rather than guessing.
+- **`score`**: use `expected_level` for thresholds ("escalate if ≥ 1.5") rather than `choice`.
+
+---
+
+## 6. Testing the model
+
+Tests 6.1–6.3 need no labelled data, so run them in CI whenever you change the revision. Test 6.4 is the one that tells you whether to ship.
+
+### 6.1 Integrity and determinism
+
+Save as `check_lightdec.py` and run `python check_lightdec.py [revision]`.
+
+```python
+import contextlib, io, sys
+import numpy as np
+from lightdec import load_lightdec
+
+rev = sys.argv[1] if len(sys.argv) > 1 else None
+log = io.StringIO()
+with contextlib.redirect_stdout(log):
+    fdm, model, tok = load_lightdec(revision=rev)
+assert "load warning" not in log.getvalue(), log.getvalue()
+
+fc = model.fcfg
+assert fc["name"] == "FalconDec", fc["name"]           # LightDec checkpoints use the FalconDec architecture name
+T = model.temperature.float().cpu().numpy()
+assert T.shape == (3, 4) and (T > 0).all(), T
+q = {"team": {"question": "Which team?", "options": ["recover password", "shipping", "invoicing"]}}
+a = fdm.decide(model, tok, "I forgot my password and can't sign in.", q)["answers"]["team"]
+b = fdm.decide(model, tok, "I forgot my password and can't sign in.", q)["answers"]["team"]
+assert all(abs(a["probs"][k] - b["probs"][k]) < 1e-4 for k in a["probs"]), "non-deterministic"
+assert abs(sum(a["probs"].values()) - 1) < 1e-3
+print(f"OK  LightDec (FalconDec v{fc['version']}, notebook {fc.get('notebook_version')})  choice={a['choice']}  "
+      f"conf={a['confidence']:.3f}  defer_threshold={fc.get('defer_threshold')}")
+```
+
+The stored temperatures should read approximately `[[1.707, 1.352, 1.466, 1.349], [1.402 ×4], [1.453 ×4]]` (§11.2).
+
+### 6.2 Behavioural tests (pytest)
+
+Save as `tests/test_lightdec.py` and run `pytest -q`. Set `LIGHTDEC_REVISION` to test a pinned commit.
+
+```python
+import os, random
+import pytest
+from lightdec import load_lightdec
+
+
+@pytest.fixture(scope="session")
+def fd():
+    return load_lightdec(revision=os.environ.get("LIGHTDEC_REVISION"))
+
+
+def ask(fd, state, question, options, **kw):
+    fdm, model, tok = fd
+    return fdm.decide(model, tok, state, [dict(question=question, options=options, **kw)])["results"][0]
+
+
+def test_probabilities_are_valid(fd):
+    r = ask(fd, "The build failed on main.", "What next?", ["Revert", "Ignore", "Retry"])
+    assert all(0 <= p <= 1 for p in r["probs"].values()) and abs(sum(r["probs"].values()) - 1) < 1e-3
+
+
+def test_typed_outputs(fd):
+    fdm, model, tok = fd
+    out = fdm.decide(model, tok, "I was charged twice. Refund me or I'm leaving.", {
+        "refund": {"type": "noul", "instructions": "Does the user ask for a refund?"},
+        "urgency": {"type": "score", "instructions": "How urgent?", "criteria": ["low", "medium", "high"]}})["answers"]
+    assert 0 <= out["refund"]["p_true"] <= 1 and out["refund"]["choice"] in (True, False)
+    assert 0 <= out["urgency"]["expected_level"] <= 2
+
+
+def test_support_routing(fd):
+    r = ask(fd, "I forgot my password and the reset email never arrived.", "Which team should handle this?",
+            ["recover password", "billing and payment", "delivery information"])
+    assert r["choice"] == "recover password"
+
+
+def test_fanout_matches_single_questions(fd):
+    # Batching changes padding; under bf16 that moves probabilities slightly, never the substance.
+    fdm, model, tok = fd
+    state = "I forgot my password and can't sign in."
+    qs = [{"question": "Team?", "options": ["recover password", "shipping", "invoicing"]},
+          {"question": "Urgent?", "options": ["Yes", "No"]}]
+    together = fdm.decide(model, tok, state, qs)["results"]
+    for q, t in zip(qs, together):
+        alone = fdm.decide(model, tok, state, [q])["results"][0]
+        assert all(abs(alone["probs"][k] - t["probs"][k]) < 2e-2 for k in alone["probs"])
+
+
+def test_option_order_is_mostly_irrelevant(fd):
+    # The head is order-equivariant, but the encoder sees positions; training reshuffled options every epoch.
+    state, q = "Where is my parcel? It's three days late.", "What should support do?"
+    opts = ["Give the delivery status", "Start a refund", "Book an appointment"]
+    base = ask(fd, state, q, opts)["choice"]
+    same = sum(ask(fd, state, q, random.Random(s).sample(opts, len(opts)))["choice"] == base for s in range(5))
+    assert same >= 4
+
+
+def test_many_options_use_the_tournament(fd):
+    opts = [f"topic number {i}" for i in range(119)] + ["reset my password"]
+    r = ask(fd, "I can't log in, I need to reset my password.", "What does the user want?", opts)
+    assert len(r["probs"]) == 120 and abs(sum(r["probs"].values()) - 1) < 1e-3
+
+
+def test_int8_agrees_with_fp16(fd):
+    fdm8, m8, tok8 = load_lightdec(revision=os.environ.get("LIGHTDEC_REVISION"), variant="int8")
+    fdm, model, tok = fd
+    items = [dict(state=s, question="Which team?", options=["billing", "shipping", "accounts", "technical"])
+             for s in ["I was double charged", "Where is my parcel?", "Change my email", "The app crashes on start",
+                       "Refund the duplicate payment", "Package never arrived", "Reset my login", "Error 500 on checkout"]]
+    a = [p.argmax() for p in fdm.score_items(model, tok, items)]
+    b = [p.argmax() for p in fdm8.score_items(m8, tok8, items)]
+    assert sum(x == y for x, y in zip(a, b)) >= len(items) - 1
+```
+
+### 6.3 Latency
+
+```python
+import time, numpy as np, torch
+from lightdec import load_lightdec
+
+fdm, model, tok = load_lightdec()
+q = [{"question": "Route?", "options": ["billing and payment", "shipping", "recover password"]}]
+for _ in range(5):
+    fdm.decide(model, tok, "I was charged twice.", q)
+t = []
+for _ in range(100):
+    if torch.cuda.is_available(): torch.cuda.synchronize()
+    t0 = time.perf_counter(); fdm.decide(model, tok, "I was charged twice.", q)
+    if torch.cuda.is_available(): torch.cuda.synchronize()
+    t.append((time.perf_counter() - t0) * 1000)
+print(f"p50 {np.percentile(t, 50):.1f} ms   p95 {np.percentile(t, 95):.1f} ms   on {model.device}")
+```
+
+For CPU serving, load the int8 variant with `device="cpu", dtype=torch.float32`, and optionally apply `torch.ao.quantization.quantize_dynamic(model, {torch.nn.Linear}, dtype=torch.qint8)` for int8 matrix multiplies.
+
+### 6.4 Accuracy on your own labelled data
+
+Write 50–500 decisions that look like your real traffic, one JSON object per line. `expected` may be a letter, a 0-based index or the option text; `type` is optional.
+
+```json
+{"id": "t1", "tag": "support", "state": "…", "question": "…", "options": ["…", "…"], "expected": "B", "type": "choice"}
+```
+
+```python
+import json, string
+import numpy as np
+from lightdec import load_lightdec
+
+fdm, model, tok = load_lightdec()
+rows = [json.loads(l) for l in open("my_eval.jsonl", encoding="utf-8") if l.strip()]
+
+def idx(v, opts):
+    if isinstance(v, int): return v
+    v = str(v).strip()
+    if len(v) == 1 and v.upper() in string.ascii_uppercase[:len(opts)]: return string.ascii_uppercase.index(v.upper())
+    return [o.lower() for o in opts].index(v.lower())
+
+items = [dict(state=r["state"], question=r.get("question", ""), options=r["options"], type=r.get("type", "choice"))
+         for r in rows]
+probs = fdm.score_items(model, tok, items, batch_size=64)
+gold = np.array([idx(r["expected"], r["options"]) for r in rows])
+pred = np.array([p.argmax() for p in probs]); conf = np.array([p.max() for p in probs]); ok = pred == gold
+
+def ece(c, k, bins=15):
+    e = 0.0
+    for lo in np.linspace(0, 1, bins, endpoint=False):
+        m = (c > lo) & (c <= lo + 1 / bins)
+        if m.any(): e += m.mean() * abs(c[m].mean() - k[m].mean())
+    return e
+
+print(f"accuracy {ok.mean():.3f} | ECE {ece(conf, ok):.3f}")
+for tag in sorted({r.get("tag", "all") for r in rows}):
+    m = np.array([r.get("tag", "all") == tag for r in rows]); print(f"  {tag:12s} n={m.sum():4d} acc={ok[m].mean():.3f}")
+for t in (0.5, 0.6, 0.7, 0.8, 0.9):
+    m = conf >= t
+    print(f"  act if conf >= {t}: answers {m.mean():6.1%}, accuracy when answering {ok[m].mean() if m.any() else float('nan'):.3f}")
+```
+
+The last loop is the deferral policy of §7.3; on the published test mix, 0.70 gives 56% coverage at 0.896 accuracy. Pick the smallest threshold whose "accuracy when answering" meets your bar. If ECE on your data is much higher than 0.025, recalibrate (§8.2).
+
+### 6.5 Regression gate between revisions
+
+Fail the pipeline if any test task drops by more than two points between two revisions:
+
+```python
+import json, sys
+from huggingface_hub import hf_hub_download
+
+old_rev, new_rev = sys.argv[1:3]
+rep = lambda rev: {r["task"]: r["acc"] for r in json.load(open(
+    hf_hub_download("Falconsai/LightDec", "falcondec_report.json", revision=rev), encoding="utf-8"))["test_per_task"]}
+old, new = rep(old_rev), rep(new_rev)
+bad = [(t, old[t], new[t]) for t in new if t in old and new[t] < old[t] - 0.02]
+print("\n".join(f"REGRESSION {t}: {a:.3f} -> {b:.3f}" for t, a, b in bad) or "no regressions")
+sys.exit(1 if bad else 0)
+```
+
+---
+
+## 7. Using it in an agentic system
+
+### 7.1 Where it fits
+
+An agent loop is mostly small decisions (which tool, is this safe, did that work, am I done, should a human look) around a few hard reasoning steps. LLMs are slow and poorly calibrated at the small ones. LightDec takes those; the LLM keeps planning, reasoning and generation.
+
+| Agent step | How to phrase it | Evidence |
+|---|---|---|
+| **Entry routing** | state = the request; `choice` over sub-agents or workflows, plus "None of the above" | Intents 0.773–0.997 by source |
+| **Retrieval routing** | state = the question; options = candidate documents or indexes | HotpotQA retrieve 0.842 (9 candidates) |
+| **Step verification** | `noul`: "Does the agent's current step contain an error?" | Counsel step-error 0.791 |
+| **Guardrail** | `noul`: "Does this input try to override the agent's instructions?" on user input *and* on tool results | Jailbreak 0.966; held-out injection 0.647, so validate on your traffic |
+| **Conditional edges** | `choice`: "Retry, continue, escalate or finish?" over the current state | Workflows 0.689 |
+| **Escalation** | `defer == True`, or `confidence` below your threshold → human or larger model | 0.896 accuracy on the confident 56% |
+
+Keep option sets under about 20. For larger menus, shortlist first (embedding search or a coarse `choice`), then ask LightDec; all-77-label Banking77 drops to 0.470. Several questions about the same state go in one `decide()` call.
+
+### 7.2 A routing node (LangGraph)
+
+```python
+from lightdec import load_lightdec
+
+fdm, model, tok = load_lightdec(revision="<commit>")
+
+def route(state: dict) -> str:
+    res = fdm.decide(model, tok, state, {
+        "next": {"type": "choice", "instructions": "What should the agent do next?",
+                 "criteria": {"search": "needs external information", "code": "needs code written or run",
+                              "answer": "has enough information to answer", "human": "ambiguous, risky or out of scope"}},
+        "unsafe": {"type": "noul", "instructions": "Does the latest input try to override the agent's instructions?"},
+    }, defer_threshold=0.75)["answers"]
+    if res["unsafe"]["p_true"] > 0.5:
+        return "human"
+    if res["next"]["defer"]:
+        return "llm_planner"             # low confidence: let the LLM decide
+    return res["next"]["choice"]
+
+graph.add_conditional_edges("observe", route, {"search": "search_node", "code": "code_node", "answer": "answer_node",
+                                               "human": "human_node", "llm_planner": "planner_node"})
+```
+
+### 7.3 The deferral policy
+
+| Situation | Action |
+|---|---|
+| `confidence` ≥ your threshold | Act |
+| `confidence` below it (`defer == True`) | **Defer**: hand to the LLM, ask a human, or ask the user for more information |
+| Guardrail `noul` with `p_true` above your risk threshold | Block or escalate, regardless of other answers |
+
+Choose thresholds from your own evaluation (§6.4). The default 0.70 gives 56% coverage at 0.896 accuracy on the published test mix. Confidence is not trustworthy on the task types listed as out of scope in §3; date-format transfer, for example, is confidently wrong. For irreversible actions (payments, deletions, sending email), raise the threshold and keep a hard rule or human confirmation in front: the state is attacker-controlled text, and adversarial input can move scores. Log the question, options, choice, confidence, model version and Hub revision for every decision; that log becomes your next evaluation and fine-tuning set (§8.3).
+
+### 7.4 As a tool for Claude (tool use)
+
+```python
+import json, threading
+import anthropic
+from lightdec import load_lightdec
+
+fdm, model, tok = load_lightdec()
+lock = threading.Lock()
+client = anthropic.Anthropic()
+tools = [{
+    "name": "lightdec_decide",
+    "description": ("Fast, local, calibrated closed-set decision model. Give it a state (text or JSON), a question and "
+                    "2-20 distinct options; it returns the choice, a calibrated confidence and a 'defer' flag. "
+                    "If 'defer' is true, don't rely on the answer. Not for arithmetic, dates or multi-step reasoning."),
+    "input_schema": {"type": "object", "properties": {
+        "state": {"type": "string", "description": "The message, document excerpt or JSON state."},
+        "question": {"type": "string"},
+        "options": {"type": "array", "items": {"type": "string"}, "minItems": 2},
+        "type": {"type": "string", "enum": ["choice", "score"], "description": "score = options are ordered levels"}},
+        "required": ["state", "question", "options"]},
+}]
+
+def run_tool(inp):
+    with lock:
+        r = fdm.decide(model, tok, inp["state"], [{"question": inp["question"], "options": inp["options"],
+                                                   "type": inp.get("type", "choice")}])["results"][0]
+    return {k: r[k] for k in ("choice", "confidence", "defer", "probs") if k in r}
+
+messages = [{"role": "user", "content": "Triage: 'I forgot my password and the reset email never arrived.' "
+                                        "Teams: Accounts, Billing, Shipping."}]
+while True:
+    resp = client.messages.create(model="claude-sonnet-5", max_tokens=1024, tools=tools, messages=messages)
+    if resp.stop_reason != "tool_use":
+        print("".join(b.text for b in resp.content if b.type == "text"))
+        break
+    messages.append({"role": "assistant", "content": resp.content})
+    results = []
+    for block in resp.content:
+        if block.type == "tool_use" and block.name == "lightdec_decide":
+            try:
+                results.append({"type": "tool_result", "tool_use_id": block.id, "content": json.dumps(run_tool(block.input))})
+            except Exception as exc:
+                results.append({"type": "tool_result", "tool_use_id": block.id, "content": str(exc), "is_error": True})
+    messages.append({"role": "user", "content": results})
+```
+
+### 7.5 As an MCP server
+
+`pip install mcp`, then save `lightdec_mcp.py` next to `lightdec.py`:
+
+```python
+import os, threading
+from mcp.server.fastmcp import FastMCP
+from lightdec import load_lightdec
+
+fdm, model, tok = load_lightdec(revision=os.environ.get("LIGHTDEC_REVISION"),
+                                variant=os.environ.get("LIGHTDEC_VARIANT", "fp16"))
+MIN_CONF = float(os.environ.get("LIGHTDEC_MIN_CONF", "0.7"))
+lock = threading.Lock()
+mcp = FastMCP("lightdec")
+
+
+@mcp.tool()
+def decide(state: str, question: str, options: list[str], type: str = "choice") -> dict:
+    """Choose one of 2-20 distinct options for a question about a state. type="score" means ordered levels.
+    Returns the choice, a calibrated confidence and 'defer' (true = not reliable enough to act on)."""
+    with lock:
+        r = fdm.decide(model, tok, state, [{"question": question, "options": options, "type": type}],
+                       defer_threshold=MIN_CONF)["results"][0]
+    return {k: r[k] for k in ("choice", "confidence", "defer", "probs", "expected_level") if k in r}
+
+
+@mcp.tool()
+def decide_many(state: str, questions: dict) -> dict:
+    """Several typed questions about one state: {name: {"type": "choice"|"noul"|"score",
+    "instructions": str, "criteria": {key: description} | [levels]}}."""
+    with lock:
+        return fdm.decide(model, tok, state, questions, defer_threshold=MIN_CONF)["answers"]
+
+
+if __name__ == "__main__":
+    mcp.run()
+```
+
+Register it in Claude Desktop's `claude_desktop_config.json`:
+
+```json
+{
+  "mcpServers": {
+    "lightdec": {
+      "command": "C:\\path\\to\\python.exe",
+      "args": ["C:\\path\\to\\lightdec_mcp.py"],
+      "env": { "LIGHTDEC_REVISION": "<commit>", "LIGHTDEC_VARIANT": "int8" }
+    }
+  }
+}
+```
+
+### 7.6 As an HTTP microservice
+
+`pip install fastapi uvicorn`, then save `serve_lightdec.py`:
+
+```python
+import threading
+from fastapi import FastAPI, HTTPException
+from pydantic import BaseModel
+from lightdec import load_lightdec
+
+fdm, model, tok = load_lightdec()
+lock = threading.Lock()
+app = FastAPI(title="LightDec")
+
+
+class Decide(BaseModel):
+    state: str | dict
+    questions: dict | list
+    defer_threshold: float = 0.7
+
+
+@app.get("/health")
+def health():
+    return {"ok": True, "model": "LightDec", "version": model.fcfg.get("version")}
+
+
+@app.post("/v1/decide")
+def decide(req: Decide):
+    try:
+        with lock:
+            return fdm.decide(model, tok, req.state, req.questions, defer_threshold=req.defer_threshold)
+    except (ValueError, KeyError, TypeError) as exc:
+        raise HTTPException(422, str(exc))
+```
+
+Run it with `uvicorn serve_lightdec:app --port 9904 --workers 1`. Each worker holds its own copy of the model; scale out with more processes.
+
+### 7.7 Without an LLM: a support intake step
+
+```python
+from lightdec import load_lightdec
+
+fdm, model, tok = load_lightdec()
+QUEUES = {"billing": "charges, invoices, payments, refunds", "returns": "returning or exchanging items",
+          "delivery": "shipping status, late or missing parcels", "accounts": "login, password, profile",
+          "human": "complaints or requests to speak to a person"}
+
+def intake(message: str) -> dict:
+    a = fdm.decide(model, tok, message, {
+        "queue": {"type": "choice", "instructions": "Which team should handle this message?", "criteria": QUEUES},
+        "urgent": {"type": "noul", "instructions": "Does this need a reply within the hour?"},
+    })["answers"]
+    if a["queue"]["defer"]:
+        return {"action": "human_review", "suggestion": a["queue"]["choice"],
+                "reason": f"low confidence ({a['queue']['confidence']:.0%})"}
+    return {"action": "enqueue", "queue": a["queue"]["choice"],
+            "priority": "high" if a["urgent"]["p_true"] >= 0.5 else "normal",
+            "evidence": {"confidence": round(a["queue"]["confidence"], 3), "model": "LightDec"}}
+```
+
+---
+
+## 8. Tuning for your domain
+
+### 8.1 Change the deferral threshold
+
+Pass `defer_threshold=` to `decide()`, or set `model.fcfg["defer_threshold"]`. This changes nothing in the model and is usually enough.
+
+### 8.2 Recalibrate on your data
+
+If ECE on your traffic (§6.4) is noticeably worse than 0.025, fit one extra temperature on top of the stored ones and save a recalibrated copy:
+
+```python
+import numpy as np, torch
+# probs, gold: from §6.4 (probabilities already include the stored temperatures)
+logp = [np.log(np.clip(p, 1e-12, 1)) for p in probs]
+def nll(s):
+    return -np.mean([(lp / s)[g] - np.log(np.exp(lp / s).sum()) for lp, g in zip(logp, gold)])
+s = min(np.linspace(0.5, 3.0, 51), key=nll)
+print("extra temperature", s)
+with torch.no_grad():
+    model.temperature.mul_(float(s))
+fdm.save_falcondec(model, tok, "lightdec_recalibrated")            # add int8=True for the compact variant
+```
+
+Fit on one labelled set and measure on another.
+
+### 8.3 Fine-tune on your own decisions
+
+Use the FalconDec training notebook (V2):
+
+1. Export your logged and corrected decisions as JSONL: `{"state", "question", "options", "answer", "type"?, "task"?}`.
+2. In cell 2, set `MODE="finetune"`, `FINETUNE_FROM="Falconsai/LightDec"`, `CUSTOM_DATA_JSONL="your_file.jsonl"`, and a preset.
+3. Run the notebook. The version bumps automatically (1.0.0 → 1.0.1), and the report records the lineage.
+
+Your tasks are up-weighted (`CUSTOM_WEIGHT`), while the public tasks keep the model general. Adding decisions with ISO-format dates and numeric tables is the most direct fix for the transfer weaknesses in §3. Check the regression gate (§6.5) before publishing.
+
+### 8.4 Save and publish
+
+```python
+fdm.save_falcondec(model, tok, "lightdec_out")                     # fp16
+fdm.save_falcondec(model, tok, "lightdec_out/compact-int8", int8=True)
+
+from huggingface_hub import HfApi                                   # needs a write token: huggingface-cli login
+HfApi().upload_folder(folder_path="lightdec_out", repo_id="Falconsai/LightDec", commit_message="LightDec v1.0.x")
+```
+
+---
+
+## 9. Architecture
+
+```
+[CLS] question [SEP] [MASK] option₁ [MASK] option₂ … [MASK] optionₖ [SEP] state [SEP]
+        │
+  Ettin-150M encoder (22 layers, hidden 768)
+        │
+  hidden state at each [MASK]  +  CLS context  +  question-type embedding
+        │
+  set transformer: 2 layers, 8 heads, no positional encoding → options attend to each other, order-equivariant
+        │
+  MLP → one logit per option → ÷ temperature[type, option-count bucket] → softmax
+```
+
+- **Option markers** (the approach Laya uses): every option is read at its own `[MASK]` token, so all options are scored in **one** pass.
+- **Question first, state last**: when the input is too long, the tail of the state is truncated, never the options.
+- **Adaptive option budget**: up to 24 tokens per option within a 192-token head budget that grows with the option count, and 2,048-token sequences above 24 options, so labels stay distinct. Above 96 options, `decide()` runs a tournament.
+- **Typed primitives**: `noul` is rendered as a neutral two-option Yes/No choice; `score` keeps its level order and reports an expected level.
+- **Calibration** lives in the model: a 3 × 4 temperature table (question type × option-count bucket: ≤2, 3–5, 6–12, >12).
+- **int8 storage**: per-output-channel symmetric int8 for every weight matrix, fp16 elsewhere, dequantised on load.
+
+---
+
+## 10. Training data
+
+**155,747 training, 13,148 validation and 17,498 test decisions** from 58 tasks. No source failed to load (`skipped_builders` is empty). Every example is a **decision**: `state`, `question`, `options`, `answer`, plus type, task and domain.
+
+| Domain | Sources | Decisions built |
+|---|---|---|
+| Support | Bitext customer support | Intent routing and category routing; 30% of messages wrapped as JSON program state |
+| Intents | CLINC150, MASSIVE (en), Banking77 *(held out)* | Intents as runtime-defined options, 2–48 per question; Banking77 also as one 77-option question |
+| Code | CodeXGLUE code-to-text (6 languages), Devign, BigCloneBench, MBPP, HumanEval *(held out)* | Language ID, code↔description, function naming, vulnerability, clones, task→solution; bug spotting against single-fault mutants **verified to fail the unit tests** |
+| Guardrails | Jailbreak classification, Civil Comments; deepset prompt-injections and AgentHarm *(held out)* | `noul` detection and refusal |
+| Agentic | Counsel (human meta-evaluations of agent-step critiques), HotpotQA, AgentTrek | Step-error and critique-quality, retrieval routing and comparison yes/no; AgentTrek next-action type and finish-now. AgentTrek loaded without errors, but none of its decisions appear in the test split, so its contribution isn't measured |
+| Workflows | LocalLLaMA/typed-decisions | `choice`/`noul`/`score` with the teacher's soft probabilities; test = the benchmark's 2,000-decision test split |
+| Policy | Synthetic, executable (in-notebook) | Return windows, approval tiers, AND/OR eligibility, overdue invoices, table look-ups, counting, SLA urgency, access control, invoice totals; the test split uses **transfer** wording, currencies and date formats |
+| Reasoning | ARC-Easy/Challenge and MMLU *(held out)*, OpenBookQA, SciQ, CommonsenseQA, QASC, HellaSwag, WinoGrande, MMLU auxiliary-train, BoolQ, GSM8K, AQuA-RAT, SNLI, MultiNLI, ANLI, SciTail | Multiple choice, yes/no, NLI, numeric answers with near-miss distractors |
+| Classification | AG News, Yelp (ordinal); DAIR Emotion and SST-5 *(held out)* | Topic; 5-level `score` sentiment |
+
+**Augmentation**: options are reshuffled every epoch (ordinal levels keep their order). In 8% of choice questions the gold answer is removed and "None of the above" becomes correct; in another 4% it is added as a distractor. **Balance**: tasks are sampled with p ∝ n^0.5 each epoch. **Leak guard**: training decisions whose (state, question) appears in validation or test were removed. Mind2Web was excluded (opt-in in the notebook). Held-out sources were never used for training, calibration or model selection.
+
+Check each dataset's card for its license before redistributing derived data. AgentHarm is used only as a held-out evaluation, in line with its intended use.
+
+---
+
+## 11. Training procedure and calibration
+
+### 11.1 Setup
+
+| Setting | Value |
+|---|---|
+| Mode / preset | `scratch` from the pretrained backbone / `standard` |
+| Data caps | ≤4,000 train, ≤300 validation, ≤300 test decisions per source split |
+| Epochs | 2 (best: epoch 2) |
+| Objective | Strictly proper scoring rules: log score + 0.5 × spherical score, + 1.0 × ranked probability score for `score` questions; soft targets (50/50 with the hard label) where a teacher distribution exists. These are the RLCD rewards, optimised with exact gradients |
+| Optional RLCD stage | Off |
+| Optimiser | AdamW (β 0.9/0.98, weight decay 0.01), encoder LR 4e-5 with layer-wise decay 0.9, head LR 3e-4, 6% warm-up, cosine decay, gradient clipping 1.0 |
+| Batching | Token-budget batches (16,384 tokens, ≤32 decisions), length-bucketed |
+| Weights kept | EMA of the weights (decay 0.999), best validation task-macro accuracy |
+| Precision | bf16 autocast, TF32 matmuls; attention `auto` (FlashAttention-2 if installed, else SDPA) |
+| Sequence | 512 tokens; 2,048 above 24 options; question ≤96 tokens; ≤24 tokens per option (more for code options) |
+| Hardware / time | NVIDIA GeForce RTX 5090 Laptop GPU · **46 minutes** |
+| Software | Python 3.14.4 · PyTorch 2.11.0+cu128 · transformers 5.17.0 |
+| Seed | 42 |
+
+| Epoch | Train loss | Train acc | Val macro | Val micro | Val NLL |
+|---|---|---|---|---|---|
+| 1 | 0.868 | 0.654 | 0.731 | 0.738 | 0.565 |
+| 2 | 0.540 | 0.802 | **0.763** | 0.775 | 0.520 |
+
+Validation was still improving at epoch 2, so a longer schedule (the `full` preset) is likely to help.
+
+### 11.2 Fitted temperatures
+
+| Question type | k ≤ 2 | k 3–5 | k 6–12 | k > 12 |
+|---|---|---|---|---|
+| choice | 1.707 | 1.352 | 1.466 | 1.349 |
+| noul | 1.402 | 1.402 | 1.402 | 1.402 |
+| score | 1.453 | 1.453 | 1.453 | 1.453 |
+
+All temperatures are above 1, so the raw model was over-confident, as Laya's checkpoints are. `noul` and `score` use a single per-type temperature because their questions fall into one option-count bucket (2 options for `noul`, mostly 3–5 levels for `score`).
+
+---
+
+## 12. Operational notes
+
+- **Concurrency.** `decide()` isn't internally locked. Serialise calls with a lock per process (as in §7.4–7.6) and scale out with processes.
+- **Hardware.** On a GPU, expect tens of milliseconds per call; measure yours with §6.3. On CPU, use the int8 variant (loaded as fp32) and optionally dynamic int8 quantisation.
+- **Input length.** 512 tokens by default. The question and options come first, so an over-long state loses its *end*. Put the decisive information early, or summarise.
+- **Determinism.** Repeated identical calls give identical results on the same hardware and library versions. Under bf16 on GPU, batching different questions together changes padding and can move probabilities slightly (about 1e-2); near-ties can flip. Decisions with a clear margin don't change.
+- **Traceability.** Log the model version, the Hub revision and the variant (fp16/int8) with every decision.
+- **Offline use.** After the first download, set `HF_HUB_OFFLINE=1`, or save a local copy and load it by path.
+
+---
+
+## 13. Bias, risks and limitations
+
+- **Quantitative and date reasoning.** Arithmetic (AQuA 0.259), table comparisons (0.290) and counting or summing (0.523) are near chance. Overdue-invoice checks with ISO dates score 0.407 with ECE 0.364, meaning confidently wrong.
+- **Wide option sets.** Accuracy falls to 0.470 with all 77 Banking77 intents in one question; shortlist first.
+- **Generalisation gap.** Held-out tasks average 0.567 against 0.725 overall. Expect lower accuracy on traffic unlike the training mix, and measure it (§6.4).
+- **Guardrail transfer.** In-distribution jailbreak detection is 0.966, but held-out prompt-injection (0.647) and AgentHarm refusal (0.654) are much lower, with ECE 0.272 and 0.178. Don't rely on it as the only safety layer.
+- **No completed head-to-head with proof_v2.** The comparisons in §2.4–2.5 use published numbers on different samples.
+- **Closed world.** The model always picks one of your options. Add "None of the above" when appropriate.
+- **Distribution-dependent calibration.** ECE was measured on this test mix; re-check it on your traffic and recalibrate if needed (§8.2).
+- **Adversarial input.** The state is untrusted text. The model can't be instructed like an LLM, but crafted input can shift its scores. Don't make it the only safeguard before irreversible actions.
+- **Data provenance.** Training data is English, largely crowd-sourced, templated, synthetic or scraped from public code and web tasks. Biases in these sources and in the backbone's pre-training can carry into decisions. Automated routing can systematically misroute users whose phrasing differs from the training data (dialects, non-native speakers, assistive phrasing); monitor misroutes by group where possible.
+- **Oversight.** Not for high-stakes decisions without human review.
+
+---
+
+## 14. Versioning and lineage
+
+| | proof_V_1 | proof_v2 | proof_v3 | **LightDec** |
+|---|---|---|---|---|
+| Model | falconsproof v1 | falconsproof v2.0.0 | FalconDec v1.0.0 (notebook V1) | **FalconDec v1.0.0 (notebook V2)** |
+| Backbone | DistilBERT, 128 tokens | ModernBERT-base, 384 tokens | Ettin-150M, 512 / 2,048 | Ettin-150M, 512 / 2,048 |
+| Preset / epochs | — | small / 1 per stage | small / 1 | **standard / 2** |
+| Training decisions | — | 18,795 | ≤500 per task | **155,747** |
+| Agentic data | — | — | — | AgentTrek, Counsel, HotpotQA |
+| Test accuracy | — | 0.813 (23-task, code-heavy mix) | 0.518 micro / 0.535 macro | **0.725 / 0.725** (58-task mix) |
+| ECE | — | 0.008 | 0.014 | 0.025 |
+| Weights | — | ≈596 MB fp32 | 319 MB fp16 | 319 MB fp16 · 161 MB int8 |
+
+LightDec is a fresh `scratch` run from the pretrained Ettin backbone (lineage: `jhu-clsp/ettin-encoder-150m`). It doesn't inherit proof_v3's or proof_v2's weights. Fine-tuning LightDec with the notebook bumps the patch version (1.0.0 → 1.0.1).
+
+**Changelog.** LightDec 1.0.0: first release. Notebook V2, `standard` preset, 2 epochs, seed 42.
+
+---
+
+## 15. API reference
+
+All functions live in `falcondec_modeling.py`.
+
+| Function | Description |
+|---|---|
+| `load_falcondec(path, device=None, dtype=None, attn_implementation="sdpa")` | Loads a FalconDec directory (fp16 or int8) or Hub repo id. Returns `(model, tokenizer)`; cuda if available |
+| `decide(model, tok, state, questions, defer_threshold=None, batch_size=32)` | Typed questions about one state. `questions` is a list or a `{key: question}` dict. Returns `{"results": [...], "answers": {key: result}}` |
+| `score_items(model, tok, items, batch_size=32)` | Batch scoring. `items = [{"state", "question", "options", "type"?, "option_tokens"?, "seq_len"?}]`. Returns a calibrated probability array per item |
+| `save_falcondec(model, tok, out_dir, int8=False, extra_files=None)` | Writes a self-contained directory (copies the modeling file) |
+| `quantize_int8(state_dict)` / `dequantize_int8(state_dict)` | The per-channel int8 codec used for `compact-int8/` |
+| `assemble(...)`, `collate_features(...)` | Low-level sequence building and batching |
+
+**Question fields**: `type` (`choice` / `noul` / `score`, default `choice`); `question` or `instructions`; `options` (list) or `criteria` (dict `{key: description}` for choice, list of levels for score); `labels` (`{"true": …, "false": …}` wording for noul); `option_tokens` and `seq_len` (optional per-question budgets).
+
+**Model attributes**: `model.fcfg` (the live config: layout, `special`, `defer_threshold`, `version`, `lineage`); `model.temperature` (3 × 4 tensor); `model.num_parameters()`; `model.device`.
+
+---
+
+## 16. Citation and references
+
+```bibtex
+@misc{falconsai_lightdec_2026,
+  title        = {LightDec: a lightweight, single-pass, typed, calibrated decision model for agentic systems},
+  author       = {{Falconsai}},
+  year         = {2026},
+  howpublished = {\url{https://huggingface.co/Falconsai/LightDec}},
+  note         = {FalconDec architecture, Ettin-150M backbone; successor to Falconsai/proof_v3}
+}
+```
+
+**Methods.** Warner et al. (2024), *ModernBERT*. Weller et al. (2025), *Ettin* encoders. Gneiting and Raftery (2007), *Strictly Proper Scoring Rules*. Guo et al. (2017), *On Calibration of Modern Neural Networks*. Geifman and El-Yaniv (2017), *Selective Classification*. Zaheer et al. (2017), *Deep Sets*; Lee et al. (2019), *Set Transformer*. Williams (1992), REINFORCE; Shao et al. (2024), GRPO. Hinton, Vinyals and Dean (2015), *Distillation*. Related decision models: Laya (Convai Innovations), TypeSafe Jev, Together Tev1.
+
+**Data.** ARC, OpenBookQA, SciQ, CommonsenseQA, QASC, HellaSwag, WinoGrande, MMLU, BoolQ, GSM8K, AQuA-RAT, SNLI, MultiNLI, ANLI, SciTail, CLINC150, MASSIVE, Bitext, Banking77, AG News, Yelp, DAIR Emotion, SST-5, jailbreak-classification, Civil Comments, deepset prompt-injections, AgentHarm, CodeXGLUE, MBPP, HumanEval, LocalLLaMA/typed-decisions, AgentTrek, Counsel, HotpotQA.
+
+Report issues, misroutes or evaluation results through the Community tab of this repository.