README.md

---
license: apache-2.0
language:
- en
library_name: transformers
base_model: jhu-clsp/ettin-encoder-150m
pipeline_tag: zero-shot-classification
tags:
- decision-model
- system-one
- falcondec
- calibrated-decisions
- multiple-choice
- intent-classification
- customer-support
- natural-language-inference
- code
- guardrails
- selective-prediction
- falconsai
- model-surgeon
- attested-lineage
---

# Falconsai/proof_v3 — FalconDec v1.0.0

**[View in Model Surgeon](https://surgeon.falcons.ai/?hub=Falconsai/proof_v3)**

**A single-pass, typed, calibrated decision model.** Give it a **state** (text, code or JSON), one or more **typed questions** (`choice`, `noul` yes/no, `score` ordinal) and a closed set of options. It returns a calibrated probability for every option, from one encoder pass per question, however many options there are.

This repository is both the **model card** and the **developer guide**: how to load it, test it, and use it as a decision component in an agentic system.

> **Read this first.** v1.0.0 is the first FalconDec checkpoint, trained with the **`small` preset** (at most 500 training decisions per task, 1 epoch). It proves the architecture and pipeline end to end, and it is **well calibrated but under-trained**: test accuracy is 0.518 (micro) / 0.535 (task-macro). Evaluate it on your own data (§6.4) before acting on its answers, and use the deferral policy (§7.3). A `standard`-preset retrain is the planned next version.

## Contents

1. [Model summary](#1-model-summary)
2. [Results](#2-results)
3. [Intended and out-of-scope uses](#3-intended-and-out-of-scope-uses)
4. [The Hub repository](#4-the-hub-repository)
5. [Install, load and read a result](#5-install-load-and-read-a-result)
6. [Testing the model](#6-testing-the-model)
7. [Using it in an agentic system](#7-using-it-in-an-agentic-system)
8. [Tuning for your domain](#8-tuning-for-your-domain)
9. [Architecture](#9-architecture)
10. [Training data](#10-training-data)
11. [Training procedure and calibration](#11-training-procedure-and-calibration)
12. [Operational notes](#12-operational-notes)
13. [Bias, risks and limitations](#13-bias-risks-and-limitations)
14. [Versioning and lineage](#14-versioning-and-lineage)
15. [API reference](#15-api-reference)
16. [Citation and references](#16-citation-and-references)

---

## 1. Model summary

| | |
|---|---|
| **Model** | FalconDec (Falcon Decision) **v1.0.0**, published as `Falconsai/proof_v3` |
| **Task** | Closed-set decisions: given a state, a question and 2–N options, return a calibrated probability per option |
| **Question types** | `choice` (pick one), `noul` (yes/no, returns P(true)), `score` (ordinal rubric, returns the expected level) |
| **Backbone** | [`jhu-clsp/ettin-encoder-150m`](https://huggingface.co/jhu-clsp/ettin-encoder-150m) (ModernBERT-style encoder), fully fine-tuned |
| **Decision head** | Option-marker scoring plus a permutation-equivariant set-transformer head (§9) |
| **Parameters** | 159.7M |
| **Weights** | fp16 **305 MB** (`model.safetensors`) · per-channel int8 **153 MB** (`compact-int8/`) |
| **Context** | 512 tokens; automatically 2,048 for questions with more than 24 options; a tournament above 96 options |
| **Calibration** | One temperature per (question type × option-count bucket), stored in the checkpoint and applied automatically |
| **Inference cost** | One encoder pass per question; **15.7 ms p50** for a single question on the training GPU |
| **Output** | Probabilities, the chosen option, confidence, a `defer` flag, `p_true` (noul) and `expected_level` (score) |
| **Language** | English, plus code in Python, Java, JavaScript, PHP, Ruby, Go and C |
| **Custom code** | `falcondec_modeling.py` ships with the weights and holds the model and all inference logic. Load it with `importlib` (§5); `AutoModel.from_pretrained` alone won't build the decision head |
| **License** | Apache-2.0 for the weights and code. Check each training dataset's license before redistributing derived data |

The model has no generative component. It can only rank the options you give it, so it cannot produce text outside that set.

---

## 2. Results

All numbers come from this checkpoint's `falcondec_report.json`: one run, `small` preset, seed 42, `MODE="scratch"`.

### 2.1 At a glance

| Metric | Value |
|---|---|
| Test accuracy, micro (all test decisions) | **0.518** |
| Test accuracy, task-macro (mean over tasks) | **0.535** |
| Held-out tasks, task-macro (9 tasks never trained on) | **0.436** |
| Expected calibration error (ECE, 15 bins) | **0.014** |
| Area under the risk–coverage curve (AURC, lower is better) | 0.270 |
| Latency, 1 question, p50 (GPU) | 15.7 ms |
| Weights | fp16 305 MB · int8 153 MB |

**Calibration is the strongest result.** An ECE of 0.014 means a stated confidence of 80% corresponds to about 80% accuracy on data like this test set. That makes confidence thresholds (§7.3) meaningful even while raw accuracy is modest: the model knows when it doesn't know.

### 2.2 Per task

The full per-task table (n, accuracy, chance, ECE, NLL, held-out flag) is in `falcondec_report.json` under `test_per_task`. Print it with:

```python
import json, pandas as pd
from huggingface_hub import hf_hub_download

rep = json.load(open(hf_hub_download("Falconsai/proof_v3", "falcondec_report.json"), encoding="utf-8"))
df = pd.DataFrame(rep["test_per_task"]).set_index("task")
print(df[["domain", "heldout", "n", "chance", "acc", "ece"]].sort_values(["domain", "acc"]).round(3).to_string())
print(rep["test_overall"])
```

The 9 held-out tasks are Banking77 (2–5 options), Banking77 with **all 77 labels in one question**, ARC-Easy, ARC-Challenge, MMLU, DAIR Emotion, SST-5 (ordinal), deepset prompt-injections and HumanEval (correct completion vs single-fault mutants).

### 2.3 Comparison with proof_v2 (not like-for-like)

| | proof_v2 | **proof_v3 (FalconDec v1.0.0)** |
|---|---|---|
| Overall test accuracy | 0.813 | 0.518 (micro) / 0.535 (macro) |
| Test mix | 23 tasks, 73% code, tasks it scored well on | ~40 tasks, balanced across 6 domains, including MMLU, ANLI, HellaSwag, WinoGrande, AQuA, typed-decisions and policy-transfer tasks |
| ECE | 0.008 | 0.014 |
| Forward passes per call | 2 | 1 per question |
| Weights | ≈596 MB (fp32) | 305 MB fp16 / 153 MB int8 |

**These overall numbers are not comparable**: the test sets differ in both tasks and difficulty. The like-for-like head-to-head (the FalconDec notebook runs proof_v2 on identical test decisions, cell 23) **did not complete in this run**, so no claim is made here about which model is more accurate on shared tasks. That comparison is the first item for the next version.

---

## 3. Intended and out-of-scope uses

### Intended

| Use | Notes |
|---|---|
| **Support, email and ticket triage** | Department (`choice`), urgency (`score`), churn or refund flags (`noul`) in one call |
| **Intent and agent routing** | Options are defined at request time, so new intents or tools need no retraining |
| **Guardrails** | Jailbreak, prompt-injection and toxicity checks on inputs and outputs |
| **Policy checks against program state** | Return windows, approval tiers, access control; trained on executable synthetic policies |
| **Selective automation** | Act on confident answers, defer the rest (§7.3); calibration makes this dependable |

### Out of scope

- **Arithmetic and multi-step reasoning** (GSM8K, AQuA): a single encoder pass can't do System-2 work. Route these to an LLM.
- **Security or correctness gating of code**: vulnerability, clone and bug detection remain weak. Don't use it to approve code.
- **Open-ended questions**: it always picks one of your options. Add "None of the above" when that is a real possibility; it was trained with that option.
- **Non-English text.**
- **High-stakes decisions without human oversight** (medical, legal, financial, employment, safety-critical).
- **Production use of v1.0.0 without your own evaluation**: see the note at the top.

---

## 4. The Hub repository

| File | Contents |
|---|---|
| `model.safetensors` | fp16 weights: encoder and decision head, plus the temperature buffer |
| `falcondec_config.json` | Layout (sequence lengths, option budgets), special-token ids, temperatures, defer threshold, version, lineage |
| `encoder/config.json` | Backbone configuration (the encoder is rebuilt from this, then the weights are loaded) |
| `tokenizer/` | Tokenizer files |
| `falcondec_modeling.py` | `FalconDec`, `load_falcondec`, `decide`, `score_items`, `save_falcondec` and the int8 codec |
| `falcondec_report.json` | Training configuration, data counts, skipped builders, history, temperatures and all test results |
| `compact-int8/` | The same model with per-channel int8 weights (`model_int8.safetensors`); a complete, self-contained FalconDec directory |
| `README.md` | This card |

**Pin a revision in production.** The repo can change, so pass a commit hash when you load.

**Model Surgeon.** If Model Surgeon processing renames the weight file in the published copy (for example to `model_edited.safetensors`), the loader in §5.1 detects it and loads the file anyway. Run the integrity test (§6.1) after every revision change to confirm the published weights behave like the trained checkpoint.

---

## 5. Install, load and read a result

```bash
pip install torch "transformers>=4.48" safetensors huggingface_hub numpy
```

### 5.1 Load

Save this helper as `proof_v3.py` next to your code. Every example below uses it.

```python
# proof_v3.py
import importlib.util, json, shutil
from pathlib import Path
from huggingface_hub import snapshot_download


def load_proof_v3(repo="Falconsai/proof_v3", revision=None, variant="fp16", device=None, dtype=None):
    """Returns (fdm, model, tokenizer). variant: "fp16" (305 MB) or "int8" (153 MB, dequantised on load)."""
    path = Path(repo) if Path(repo).exists() else Path(snapshot_download(repo, revision=revision))
    if variant == "int8":
        path = path / "compact-int8"
    fc = json.loads((path / "falcondec_config.json").read_text(encoding="utf-8"))
    expected = fc.get("weights", "model.safetensors")
    if not (path / expected).exists():                  # e.g. a renamed file after Model Surgeon processing
        cands = sorted(p for p in path.glob("*.safetensors"))
        if not cands:
            raise FileNotFoundError(f"no .safetensors weights in {path}")
        local = Path("proof_v3_local") / variant
        shutil.copytree(path, local, dirs_exist_ok=True)
        shutil.copy(cands[0], local / expected)
        path = local
    spec = importlib.util.spec_from_file_location("falcondec_modeling", str(path / "falcondec_modeling.py"))
    fdm = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(fdm)
    model, tok = fdm.load_falcondec(str(path), device=device, dtype=dtype)   # cuda if available, else cpu
    return fdm, model, tok
```

```python
from proof_v3 import load_proof_v3

fdm, model, tok = load_proof_v3()                      # or load_proof_v3(revision="<commit>"), variant="int8"
print(model.fcfg["name"], model.fcfg["version"], model.num_parameters() / 1e6, "M params")
```

If loading prints `[FalconDec] load warning: missing=… unexpected=…`, the weights didn't match the architecture. Treat that as a failed load (§6.1 checks for it).

### 5.2 Decide

`decide()` takes one state and any number of typed questions: either a list, or a Jev/Laya-style dict keyed by name.

```python
state = {"from": "user@acme.com", "subject": "Duplicate charge on invoice #4411",
         "body": "We were billed twice for March. Please refund the duplicate today or we will cancel our plan."}

out = fdm.decide(model, tok, state, {
    "department": {"type": "choice", "instructions": "Which department should handle this request?",
                   "criteria": {"billing": "invoices, payments, refunds", "technical": "bugs, outages",
                                "sales": "pricing, contracts", "other": "everything else"}},
    "urgency":    {"type": "score", "instructions": "How urgent is this request?",
                   "criteria": ["not urgent", "soon", "critical deadline or blocking issue"]},
    "churn_risk": {"type": "noul", "instructions": "Does the user threaten to cancel or leave?"},
})
a = out["answers"]
print(a["department"]["choice"], round(a["department"]["confidence"], 3), a["department"]["defer"])
print("urgency level", round(a["urgency"]["expected_level"], 2), "of", 2)
print("P(churn)", round(a["churn_risk"]["p_true"], 3))
```

Plain options work too: `{"question": "Which team?", "options": ["Accounts", "Billing", "Shipping"]}`.

### 5.3 Reading a result

Each item in `out["results"]` (and `out["answers"][key]`) contains:

| Field | Meaning |
|---|---|
| `key` | The question's name (dict input) or `None` |
| `type` | `choice`, `noul` or `score` |
| `choice` | The chosen key: the criteria key, the option text, `True`/`False` for `noul`, or the level index for `score` |
| `choice_text` | The option text the model saw |
| `confidence` | Calibrated probability of `choice` |
| `probs` | The full distribution, keyed by `str(key)` |
| `defer` | `True` when `confidence` is below the defer threshold (default 0.70, stored in the config): don't act on it (§7.3) |
| `p_true` | `noul` only: calibrated P(yes) |
| `expected_level` | `score` only: probability-weighted level (0 … k−1); better than the argmax for ordinal rubrics |

How to read them:

- **Low confidence, spread probabilities**: the state doesn't support any option clearly. Defer, or add "None of the above".
- **`noul` near 0.5**: genuinely ambiguous. Ask for more information rather than guessing.
- **`score`**: use `expected_level` for thresholds ("escalate if ≥ 1.5") rather than `choice`.

---

## 6. Testing the model

Tests 6.1–6.3 need no labelled data, so run them in CI whenever you change the revision. Test 6.4 is the one that tells you whether to ship.

### 6.1 Integrity and determinism

Save as `check_proof_v3.py` and run `python check_proof_v3.py [revision]`.

```python
import contextlib, io, sys
import numpy as np
from proof_v3 import load_proof_v3

rev = sys.argv[1] if len(sys.argv) > 1 else None
log = io.StringIO()
with contextlib.redirect_stdout(log):
    fdm, model, tok = load_proof_v3(revision=rev)
assert "load warning" not in log.getvalue(), log.getvalue()

fc = model.fcfg
assert fc["name"] == "FalconDec", fc["name"]
T = model.temperature.float().cpu().numpy()
assert T.shape == (3, 4) and (T > 0).all(), T
q = {"team": {"question": "Which team?", "options": ["recover password", "shipping", "invoicing"]}}
a = fdm.decide(model, tok, "I forgot my password and can't sign in.", q)["answers"]["team"]
b = fdm.decide(model, tok, "I forgot my password and can't sign in.", q)["answers"]["team"]
assert all(abs(a["probs"][k] - b["probs"][k]) < 1e-4 for k in a["probs"]), "non-deterministic"
assert abs(sum(a["probs"].values()) - 1) < 1e-3
print(f"OK  FalconDec v{fc['version']}  choice={a['choice']}  conf={a['confidence']:.3f}  "
      f"defer_threshold={fc.get('defer_threshold')}  temperatures={np.round(T, 3).tolist()}")
```

### 6.2 Behavioural tests (pytest)

Save as `tests/test_proof_v3.py` and run `pytest -q`. Set `PROOF_V3_REVISION` to test a pinned commit.

```python
import os, random
import pytest
from proof_v3 import load_proof_v3


@pytest.fixture(scope="session")
def fd():
    return load_proof_v3(revision=os.environ.get("PROOF_V3_REVISION"))


def ask(fd, state, question, options, **kw):
    fdm, model, tok = fd
    return fdm.decide(model, tok, state, [dict(question=question, options=options, **kw)])["results"][0]


def test_probabilities_are_valid(fd):
    r = ask(fd, "The build failed on main.", "What next?", ["Revert", "Ignore", "Retry"])
    assert all(0 <= p <= 1 for p in r["probs"].values()) and abs(sum(r["probs"].values()) - 1) < 1e-3


def test_typed_outputs(fd):
    fdm, model, tok = fd
    out = fdm.decide(model, tok, "I was charged twice. Refund me or I'm leaving.", {
        "refund": {"type": "noul", "instructions": "Does the user ask for a refund?"},
        "urgency": {"type": "score", "instructions": "How urgent?", "criteria": ["low", "medium", "high"]}})["answers"]
    assert 0 <= out["refund"]["p_true"] <= 1 and out["refund"]["choice"] in (True, False)
    assert 0 <= out["urgency"]["expected_level"] <= 2


def test_fanout_matches_single_questions(fd):
    # Batching changes padding; under bf16 that moves probabilities slightly, never the substance.
    fdm, model, tok = fd
    state = "I forgot my password and can't sign in."
    qs = [{"question": "Team?", "options": ["recover password", "shipping", "invoicing"]},
          {"question": "Urgent?", "options": ["Yes", "No"]}]
    together = fdm.decide(model, tok, state, qs)["results"]
    for q, t in zip(qs, together):
        alone = fdm.decide(model, tok, state, [q])["results"][0]
        assert all(abs(alone["probs"][k] - t["probs"][k]) < 2e-2 for k in alone["probs"])


def test_option_order_is_mostly_irrelevant(fd):
    # The head is order-equivariant, but the encoder sees positions; training reshuffled options every epoch.
    state, q = "Where is my parcel? It's three days late.", "What should support do?"
    opts = ["Give the delivery status", "Start a refund", "Book an appointment"]
    base = ask(fd, state, q, opts)["choice"]
    same = sum(ask(fd, state, q, random.Random(s).sample(opts, len(opts)))["choice"] == base for s in range(5))
    assert same >= 4


def test_many_options_use_the_tournament(fd):
    opts = [f"topic number {i}" for i in range(119)] + ["reset my password"]
    r = ask(fd, "I can't log in, I need to reset my password.", "What does the user want?", opts)
    assert len(r["probs"]) == 120 and abs(sum(r["probs"].values()) - 1) < 1e-3


def test_int8_agrees_with_fp16(fd):
    fdm8, m8, tok8 = load_proof_v3(revision=os.environ.get("PROOF_V3_REVISION"), variant="int8")
    fdm, model, tok = fd
    items = [dict(state=s, question="Which team?", options=["billing", "shipping", "accounts", "technical"])
             for s in ["I was double charged", "Where is my parcel?", "Change my email", "The app crashes on start",
                       "Refund the duplicate payment", "Package never arrived", "Reset my login", "Error 500 on checkout"]]
    a = [p.argmax() for p in fdm.score_items(model, tok, items)]
    b = [p.argmax() for p in fdm8.score_items(m8, tok8, items)]
    assert sum(x == y for x, y in zip(a, b)) >= len(items) - 1
```

### 6.3 Latency

```python
import time, numpy as np, torch
from proof_v3 import load_proof_v3

fdm, model, tok = load_proof_v3()
q = [{"question": "Route?", "options": ["billing and payment", "shipping", "recover password"]}]
for _ in range(5):
    fdm.decide(model, tok, "I was charged twice.", q)
t = []
for _ in range(100):
    if torch.cuda.is_available(): torch.cuda.synchronize()
    t0 = time.perf_counter(); fdm.decide(model, tok, "I was charged twice.", q)
    if torch.cuda.is_available(): torch.cuda.synchronize()
    t.append((time.perf_counter() - t0) * 1000)
print(f"p50 {np.percentile(t, 50):.1f} ms   p95 {np.percentile(t, 95):.1f} ms   on {model.device}")
```

For CPU serving, load the int8 variant with `device="cpu", dtype=torch.float32`, and optionally apply `torch.ao.quantization.quantize_dynamic(model, {torch.nn.Linear}, dtype=torch.qint8)` for int8 matrix multiplies.

### 6.4 Accuracy on your own labelled data

Write 50–500 decisions that look like your real traffic, one JSON object per line. `expected` may be a letter, a 0-based index or the option text; `type` is optional.

```json
{"id": "t1", "tag": "support", "state": "…", "question": "…", "options": ["…", "…"], "expected": "B", "type": "choice"}
```

```python
import json, string
import numpy as np
from proof_v3 import load_proof_v3

fdm, model, tok = load_proof_v3()
rows = [json.loads(l) for l in open("my_eval.jsonl", encoding="utf-8") if l.strip()]

def idx(v, opts):
    if isinstance(v, int): return v
    v = str(v).strip()
    if len(v) == 1 and v.upper() in string.ascii_uppercase[:len(opts)]: return string.ascii_uppercase.index(v.upper())
    return [o.lower() for o in opts].index(v.lower())

items = [dict(state=r["state"], question=r.get("question", ""), options=r["options"], type=r.get("type", "choice"))
         for r in rows]
probs = fdm.score_items(model, tok, items, batch_size=64)
gold = np.array([idx(r["expected"], r["options"]) for r in rows])
pred = np.array([p.argmax() for p in probs]); conf = np.array([p.max() for p in probs]); ok = pred == gold

def ece(c, k, bins=15):
    e = 0.0
    for lo in np.linspace(0, 1, bins, endpoint=False):
        m = (c > lo) & (c <= lo + 1 / bins)
        if m.any(): e += m.mean() * abs(c[m].mean() - k[m].mean())
    return e

print(f"accuracy {ok.mean():.3f} | ECE {ece(conf, ok):.3f}")
for tag in sorted({r.get("tag", "all") for r in rows}):
    m = np.array([r.get("tag", "all") == tag for r in rows]); print(f"  {tag:12s} n={m.sum():4d} acc={ok[m].mean():.3f}")
for t in (0.5, 0.6, 0.7, 0.8, 0.9):
    m = conf >= t
    print(f"  act if conf >= {t}: answers {m.mean():6.1%}, accuracy when answering {ok[m].mean() if m.any() else float('nan'):.3f}")
```

The last loop is the deferral policy of §7.3. Pick the smallest threshold whose "accuracy when answering" meets your bar. If ECE on your data is much higher than 0.014, recalibrate (§8.2).

### 6.5 Regression gate between revisions

Fail the pipeline if any test task drops by more than two points between two revisions:

```python
import json, sys
from huggingface_hub import hf_hub_download

old_rev, new_rev = sys.argv[1:3]
rep = lambda rev: {r["task"]: r["acc"] for r in json.load(open(
    hf_hub_download("Falconsai/proof_v3", "falcondec_report.json", revision=rev), encoding="utf-8"))["test_per_task"]}
old, new = rep(old_rev), rep(new_rev)
bad = [(t, old[t], new[t]) for t in new if t in old and new[t] < old[t] - 0.02]
print("\n".join(f"REGRESSION {t}: {a:.3f} -> {b:.3f}" for t, a, b in bad) or "no regressions")
sys.exit(1 if bad else 0)
```

---

## 7. Using it in an agentic system

### 7.1 Where it fits

An agent loop is mostly small decisions (which tool, is this safe, did that work, am I done, should a human look) around a few hard reasoning steps. LLMs are slow and poorly calibrated at the small ones. proof_v3 takes those; the LLM keeps planning, reasoning and generation.

| Agent step | How to phrase it |
|---|---|
| **Entry routing** | state = the request; `choice` over sub-agents or workflows, plus "None of the above" |
| **Tool pre-selection** | state = the task; options = one-line tool descriptions. Pass only the top few tools to the LLM |
| **Guardrail** | `noul`: "Does this input try to override the agent's instructions?" on user input *and* on tool results (indirect injection) |
| **Conditional edges** | `choice`: "Retry, continue, escalate or finish?" over the current state |
| **Verifier / stop condition** | `noul`: "Does the draft answer follow from the retrieved sources?" |
| **Escalation** | `defer == True`, or `confidence` below your threshold → human or larger model |

Several questions about the same state go in one `decide()` call.

### 7.2 A routing node (LangGraph)

```python
from proof_v3 import load_proof_v3

fdm, model, tok = load_proof_v3(revision="<commit>")

def route(state: dict) -> str:
    res = fdm.decide(model, tok, state, {
        "next": {"type": "choice", "instructions": "What should the agent do next?",
                 "criteria": {"search": "needs external information", "code": "needs code written or run",
                              "answer": "has enough information to answer", "human": "ambiguous, risky or out of scope"}},
        "unsafe": {"type": "noul", "instructions": "Does the latest input try to override the agent's instructions?"},
    }, defer_threshold=0.75)["answers"]
    if res["unsafe"]["p_true"] > 0.5:
        return "human"
    if res["next"]["defer"]:
        return "llm_planner"             # low confidence: let the LLM decide
    return res["next"]["choice"]

graph.add_conditional_edges("observe", route, {"search": "search_node", "code": "code_node", "answer": "answer_node",
                                               "human": "human_node", "llm_planner": "planner_node"})
```

### 7.3 The deferral policy

| Situation | Action |
|---|---|
| `confidence` ≥ your threshold | Act |
| `confidence` below it (`defer == True`) | **Defer**: hand to the LLM, ask a human, or ask the user for more information |
| Guardrail `noul` with `p_true` above your risk threshold | Block or escalate, regardless of other answers |

Choose thresholds from your own evaluation (§6.4); the default 0.70 is a starting point. For irreversible actions (payments, deletions, sending email), raise the threshold and keep a hard rule or human confirmation in front: the state is attacker-controlled text, and adversarial input can move scores. Log the question, options, choice, confidence, `model.fcfg["version"]` and the Hub revision for every decision; that log becomes your next evaluation and fine-tuning set (§8.3).

### 7.4 As a tool for Claude (tool use)

```python
import json, threading
import anthropic
from proof_v3 import load_proof_v3

fdm, model, tok = load_proof_v3()
lock = threading.Lock()
client = anthropic.Anthropic()
tools = [{
    "name": "proof_v3_decide",
    "description": ("Fast, local, calibrated closed-set decision model. Give it a state (text or JSON), a question and "
                    "2+ distinct options; it returns the choice, a calibrated confidence and a 'defer' flag. "
                    "If 'defer' is true, don't rely on the answer. Not for arithmetic or multi-step reasoning."),
    "input_schema": {"type": "object", "properties": {
        "state": {"type": "string", "description": "The message, document excerpt or JSON state."},
        "question": {"type": "string"},
        "options": {"type": "array", "items": {"type": "string"}, "minItems": 2},
        "type": {"type": "string", "enum": ["choice", "score"], "description": "score = options are ordered levels"}},
        "required": ["state", "question", "options"]},
}]

def run_tool(inp):
    with lock:
        r = fdm.decide(model, tok, inp["state"], [{"question": inp["question"], "options": inp["options"],
                                                   "type": inp.get("type", "choice")}])["results"][0]
    return {k: r[k] for k in ("choice", "confidence", "defer", "probs") if k in r}

messages = [{"role": "user", "content": "Triage: 'I forgot my password and the reset email never arrived.' "
                                        "Teams: Accounts, Billing, Shipping."}]
while True:
    resp = client.messages.create(model="claude-sonnet-5", max_tokens=1024, tools=tools, messages=messages)
    if resp.stop_reason != "tool_use":
        print("".join(b.text for b in resp.content if b.type == "text"))
        break
    messages.append({"role": "assistant", "content": resp.content})
    results = []
    for block in resp.content:
        if block.type == "tool_use" and block.name == "proof_v3_decide":
            try:
                results.append({"type": "tool_result", "tool_use_id": block.id, "content": json.dumps(run_tool(block.input))})
            except Exception as exc:
                results.append({"type": "tool_result", "tool_use_id": block.id, "content": str(exc), "is_error": True})
    messages.append({"role": "user", "content": results})
```

### 7.5 As an MCP server

`pip install mcp`, then save `proof_v3_mcp.py` next to `proof_v3.py`:

```python
import os, threading
from mcp.server.fastmcp import FastMCP
from proof_v3 import load_proof_v3

fdm, model, tok = load_proof_v3(revision=os.environ.get("PROOF_V3_REVISION"),
                                variant=os.environ.get("PROOF_V3_VARIANT", "fp16"))
MIN_CONF = float(os.environ.get("PROOF_V3_MIN_CONF", "0.7"))
lock = threading.Lock()
mcp = FastMCP("proof_v3")


@mcp.tool()
def decide(state: str, question: str, options: list[str], type: str = "choice") -> dict:
    """Choose one of 2+ distinct options for a question about a state. type="score" means ordered levels.
    Returns the choice, a calibrated confidence and 'defer' (true = not reliable enough to act on)."""
    with lock:
        r = fdm.decide(model, tok, state, [{"question": question, "options": options, "type": type}],
                       defer_threshold=MIN_CONF)["results"][0]
    return {k: r[k] for k in ("choice", "confidence", "defer", "probs", "expected_level") if k in r}


@mcp.tool()
def decide_many(state: str, questions: dict) -> dict:
    """Several typed questions about one state: {name: {"type": "choice"|"noul"|"score",
    "instructions": str, "criteria": {key: description} | [levels]}}."""
    with lock:
        return fdm.decide(model, tok, state, questions, defer_threshold=MIN_CONF)["answers"]


if __name__ == "__main__":
    mcp.run()
```

Register it in Claude Desktop's `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "proof_v3": {
      "command": "C:\\path\\to\\python.exe",
      "args": ["C:\\path\\to\\proof_v3_mcp.py"],
      "env": { "PROOF_V3_REVISION": "<commit>", "PROOF_V3_VARIANT": "int8" }
    }
  }
}
```

### 7.6 As an HTTP microservice

`pip install fastapi uvicorn`, then save `serve_proof_v3.py`:

```python
import threading
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field
from proof_v3 import load_proof_v3

fdm, model, tok = load_proof_v3()
lock = threading.Lock()
app = FastAPI(title="proof_v3")


class Decide(BaseModel):
    state: str | dict
    questions: dict | list
    defer_threshold: float = 0.7


@app.get("/health")
def health():
    return {"ok": True, "model": model.fcfg.get("name"), "version": model.fcfg.get("version")}


@app.post("/v1/decide")
def decide(req: Decide):
    try:
        with lock:
            return fdm.decide(model, tok, req.state, req.questions, defer_threshold=req.defer_threshold)
    except (ValueError, KeyError, TypeError) as exc:
        raise HTTPException(422, str(exc))
```

Run it with `uvicorn serve_proof_v3:app --port 9903 --workers 1`. Each worker holds its own copy of the model; scale out with more processes.

### 7.7 Without an LLM: a support intake step

```python
from proof_v3 import load_proof_v3

fdm, model, tok = load_proof_v3()
QUEUES = {"billing": "charges, invoices, payments, refunds", "returns": "returning or exchanging items",
          "delivery": "shipping status, late or missing parcels", "accounts": "login, password, profile",
          "human": "complaints or requests to speak to a person"}

def intake(message: str) -> dict:
    a = fdm.decide(model, tok, message, {
        "queue": {"type": "choice", "instructions": "Which team should handle this message?", "criteria": QUEUES},
        "urgent": {"type": "noul", "instructions": "Does this need a reply within the hour?"},
    })["answers"]
    if a["queue"]["defer"]:
        return {"action": "human_review", "suggestion": a["queue"]["choice"],
                "reason": f"low confidence ({a['queue']['confidence']:.0%})"}
    return {"action": "enqueue", "queue": a["queue"]["choice"],
            "priority": "high" if a["urgent"]["p_true"] >= 0.5 else "normal",
            "evidence": {"confidence": round(a["queue"]["confidence"], 3), "version": model.fcfg["version"]}}
```

---

## 8. Tuning for your domain

### 8.1 Change the deferral threshold

Pass `defer_threshold=` to `decide()`, or set `model.fcfg["defer_threshold"]`. This changes nothing in the model and is usually enough.

### 8.2 Recalibrate on your data

If ECE on your traffic (§6.4) is noticeably worse than 0.014, fit one extra temperature on top of the stored ones and save a recalibrated copy:

```python
import numpy as np, torch
# probs, gold: from §6.4 (probabilities already include the stored temperatures)
logp = [np.log(np.clip(p, 1e-12, 1)) for p in probs]
def nll(s):
    return -np.mean([(lp / s)[g] - np.log(np.exp(lp / s).sum()) for lp, g in zip(logp, gold)])
s = min(np.linspace(0.5, 3.0, 51), key=nll)
print("extra temperature", s)
with torch.no_grad():
    model.temperature.mul_(float(s))
fdm.save_falcondec(model, tok, "proof_v3_recalibrated")            # add int8=True for the compact variant
```

Fit on one labelled set and measure on another.

### 8.3 Fine-tune on your own decisions

Use the FalconDec training notebook (V2):

1. Export your logged and corrected decisions as JSONL: `{"state", "question", "options", "answer", "type"?, "task"?}`.
2. In cell 2, set `MODE="finetune"`, `FINETUNE_FROM="Falconsai/proof_v3"`, `CUSTOM_DATA_JSONL="your_file.jsonl"`, and a preset.
3. Run the notebook. The version bumps automatically (1.0.0 → 1.0.1), and the report records the lineage.

Your tasks are up-weighted (`CUSTOM_WEIGHT`), while the public tasks keep the model general. Check the regression gate (§6.5) before publishing.

### 8.4 Save and publish

```python
fdm.save_falcondec(model, tok, "proof_v3_out")                     # fp16
fdm.save_falcondec(model, tok, "proof_v3_out/compact-int8", int8=True)

from huggingface_hub import HfApi                                   # needs a write token: huggingface-cli login
HfApi().upload_folder(folder_path="proof_v3_out", repo_id="Falconsai/proof_v3", commit_message="FalconDec v1.0.x")
```

---

## 9. Architecture

```
[CLS] question [SEP] [MASK] option₁ [MASK] option₂ … [MASK] optionₖ [SEP] state [SEP]
        │
  Ettin-150M encoder (22 layers, hidden 768)
        │
  hidden state at each [MASK]  +  CLS context  +  question-type embedding
        │
  set transformer: 2 layers, no positional encoding → options attend to each other, order-equivariant
        │
  MLP → one logit per option → ÷ temperature[type, option-count bucket] → softmax
```

- **Option markers** (the approach Laya uses): every option is read at its own `[MASK]` token, so all options are scored in **one** pass. proof_v2 needed one cross-encoder pair per option plus a second pass.
- **Question first, state last**: when the input is too long, the tail of the state is truncated, never the options.
- **Adaptive option budget**: the per-option token budget shrinks as the number of options grows, and the sequence stretches to 2,048 tokens above 24 options so labels stay distinct (Laya's documented Banking77 failure was 3–4 tokens per label). Above 96 options, `decide()` runs a tournament: best options of each chunk, then a final pass.
- **Typed primitives**: `noul` is rendered as a neutral two-option Yes/No choice (avoiding Laya's documented failure where the answer followed the `true:`/`false:` labels); `score` keeps its level order and reports an expected level.
- **Calibration** lives in the model: a 3 × 4 temperature table (question type × option-count bucket: ≤2, 3–5, 6–12, >12).
- **int8 storage**: per-output-channel symmetric int8 for every weight matrix, fp16 elsewhere, dequantised on load. The compute path is identical to fp16.

---

## 10. Training data

Every example is a **decision**: `state`, `question`, `options`, `answer`, plus type, task and domain. Options are reshuffled every epoch (ordinal levels keep their order). Each source was loaded independently; any source that failed to load on the training machine was skipped and is listed under `skipped_builders` in `falcondec_report.json`.

| Domain | Sources | Decisions built |
|---|---|---|
| Reasoning | ARC-Easy/Challenge (held out), OpenBookQA, SciQ, CommonsenseQA, QASC, HellaSwag, WinoGrande, MMLU auxiliary-train (MMLU test held out), BoolQ, GSM8K, AQuA-RAT | Multiple choice, yes/no, numeric answers with near-miss distractors |
| NLI | SNLI, MultiNLI, ANLI, SciTail | Three-way NLI; "which statement must be true / contradicts"; supported yes/no |
| Intents and support | CLINC150, MASSIVE (en), Bitext customer support, Banking77 (held out) | Intents as runtime-defined options, 2–48 per question; Bitext category routing; 30% of messages wrapped as JSON program state |
| Classification | AG News, Yelp (ordinal), DAIR Emotion (held out), SST-5 (held out, ordinal) | Topic; 5-level `score` sentiment |
| Guardrails | Jailbreak classification, Civil Comments, deepset prompt-injections (held out) | `noul` detection |
| Code | CodeXGLUE code-to-text (6 languages), Devign, BigCloneBench, MBPP, HumanEval (held out) | Language ID, code↔description, function naming, vulnerability, clones, task→solution; bug spotting against single-fault mutants **verified to fail the unit tests** |
| Workflows | LocalLLaMA/typed-decisions | `choice`/`noul`/`score` with the teacher's soft probabilities |
| Policy | Synthetic, executable (in-notebook) | Return windows, approval tiers, AND/OR eligibility, overdue invoices, table look-ups, counting, SLA urgency, access control, invoice totals; the test split uses *transfer* wording, currencies and date formats |

**Augmentation**: in 8% of choice questions the gold answer is removed and "None of the above" becomes correct; in another 4% it is added as a distractor. **Balance**: tasks are sampled with p ∝ n^0.5 every epoch, so no domain dominates (proof_v2 was 73% code). **Leak guard**: training decisions whose (state, question) appears in validation or test were removed.

Held-out sources were never used for training, calibration or model selection. Check each dataset's card for its license before redistributing derived data.

---

## 11. Training procedure and calibration

| Setting | Value |
|---|---|
| Mode / preset | `scratch` from the pretrained backbone / `small` |
| Data caps | ≤500 train, ≤120 validation, ≤120 test decisions per source split |
| Epochs | 1 |
| Objective | Strictly proper scoring rules: log score + 0.5 × spherical score, + ranked probability score for `score` questions; soft targets (50/50 with the hard label) where a teacher distribution exists. These are the RLCD rewards, optimised with exact gradients |
| Optional RLCD stage | Off in this run |
| Optimiser | AdamW (β 0.9/0.98, weight decay 0.01), encoder LR 4e-5 with layer-wise decay 0.9, head LR 3e-4, 6% warm-up, cosine decay, gradient clipping 1.0 |
| Batching | Token-budget batches (16,384 tokens, ≤32 decisions), length-bucketed |
| Weights kept | EMA of the weights (decay 0.999), best validation task-macro accuracy |
| Precision | bf16 autocast, TF32 matmuls |
| Sequence | 512 tokens; 2,048 above 24 options; question ≤96 tokens; ≤24 tokens per option (more for code options) |
| Calibration | Temperatures fitted with L-BFGS on validation logits per (type × option-count bucket); cells with too few examples fall back to the per-type, then the global temperature |
| Seed | 42 |

Training time, GPU and library versions are recorded in `falcondec_report.json` (`train_minutes`, `env`). Estimate emissions with the [ML CO2 Impact calculator](https://mlco2.github.io/impact/) from those values.

---

## 12. Operational notes

- **Concurrency.** `decide()` isn't internally locked. Serialise calls with a lock per process (as in §7.4–7.6) and scale out with processes.
- **Hardware.** A GPU gives ~16 ms per single-question call. On CPU, use the int8 variant (loaded as fp32) and optionally dynamic int8 quantisation; expect tens to a few hundred milliseconds depending on input length and cores.
- **Input length.** 512 tokens by default. The question and options come first, so an over-long state loses its *end*. Put the decisive information early, or summarise.
- **Determinism.** Repeated identical calls give identical results on the same hardware and library versions. Under bf16 on GPU, batching different questions together changes padding and can move probabilities slightly (about 1e-2); near-ties can flip. Decisions with a clear margin don't change.
- **Traceability.** Log `model.fcfg["version"]`, the Hub revision and the variant (fp16/int8) with every decision.
- **Offline use.** After the first download, set `HF_HUB_OFFLINE=1`, or save a local copy and load it by path.

---

## 13. Bias, risks and limitations

- **Under-trained first version.** Trained with the `small` preset (≤500 decisions per task, 1 epoch). Accuracy on reasoning-heavy and held-out tasks is modest (held-out macro 0.436).
- **No completed head-to-head with proof_v2.** Don't assume v1.0.0 is more accurate than proof_v2 on proof_v2's strong tasks until that comparison is published.
- **Closed world.** The model always picks one of your options. Add "None of the above" when appropriate.
- **Arithmetic and code correctness** remain weak; route them elsewhere.
- **Distribution-dependent calibration.** ECE was measured on this test mix; re-check it on your traffic (§6.4) and recalibrate if needed (§8.2).
- **Adversarial input.** The state is untrusted text. The model can't be instructed like an LLM, but crafted input can shift its scores. Don't make it the only safeguard before irreversible actions.
- **Data provenance.** Training data is English, largely crowd-sourced, templated, synthetic or scraped from public code. Biases in these sources and in the backbone's pre-training can carry into decisions. Automated routing can systematically misroute users whose phrasing differs from the training data (dialects, non-native speakers, assistive phrasing); monitor misroutes by group where possible.
- **Oversight.** Not for high-stakes decisions without human review.

---

## 14. Versioning and lineage

| | proof_V_1 | proof_v2 | **proof_v3** |
|---|---|---|---|
| Model | falconsproof v1 | falconsproof v2.0.0 | **FalconDec v1.0.0** |
| Backbone | DistilBERT, 128 tokens | ModernBERT-base, 384 tokens | Ettin-150M, 512 / 2,048 tokens |
| Decision design | Specialist + generalist behind a gate | 3 specialists + generalist, gates closed | Single typed head, options scored in one pass |
| Question types | choice | choice | choice, noul, score |
| Weights | — | ≈596 MB fp32 | 305 MB fp16 / 153 MB int8 |

proof_v3 is a new architecture trained from the pretrained Ettin backbone (`MODE="scratch"`); it doesn't inherit proof_v2's weights. FalconDec versions follow their own numbering: fine-tuning a FalconDec checkpoint bumps the patch version (1.0.0 → 1.0.1); a new scratch run bumps the major version.

**Changelog.** v1.0.0: first release, `small` preset, seed 42.

---

## 15. API reference

All functions live in `falcondec_modeling.py`.

| Function | Description |
|---|---|
| `load_falcondec(path, device=None, dtype=None, attn_implementation="sdpa")` | Loads a FalconDec directory (fp16 or int8) or Hub repo id. Returns `(model, tokenizer)`; cuda if available |
| `decide(model, tok, state, questions, defer_threshold=None, batch_size=32)` | Typed questions about one state. `questions` is a list or a `{key: question}` dict. Returns `{"results": [...], "answers": {key: result}}` |
| `score_items(model, tok, items, batch_size=32)` | Batch scoring. `items = [{"state", "question", "options", "type"?, "option_tokens"?, "seq_len"?}]`. Returns a calibrated probability array per item |
| `save_falcondec(model, tok, out_dir, int8=False, extra_files=None)` | Writes a self-contained FalconDec directory (copies the modeling file) |
| `quantize_int8(state_dict)` / `dequantize_int8(state_dict)` | The per-channel int8 codec used for `compact-int8/` |
| `assemble(...)`, `collate_features(...)` | Low-level sequence building and batching |

**Question fields**: `type` (`choice` / `noul` / `score`, default `choice`); `question` or `instructions`; `options` (list) or `criteria` (dict `{key: description}` for choice, list of levels for score); `labels` (`{"true": …, "false": …}` wording for noul); `option_tokens` and `seq_len` (optional per-question budgets).

**Model attributes**: `model.fcfg` (the live config: layout, `special`, `defer_threshold`, `version`, `lineage`); `model.temperature` (3 × 4 tensor); `model.num_parameters()`; `model.device`.

---

## 16. Citation and references

```bibtex
@misc{falconsai_proof_v3_2026,
  title        = {proof_v3 (FalconDec v1.0.0): a single-pass, typed, calibrated decision model},
  author       = {{Falconsai}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/Falconsai/proof_v3}},
  note         = {Ettin-150M backbone; successor to Falconsai/proof_v2}
}
```

**Methods.** Warner et al. (2024), *ModernBERT*. Weller et al. (2025), *Ettin* encoders. Gneiting and Raftery (2007), *Strictly Proper Scoring Rules*. Guo et al. (2017), *On Calibration of Modern Neural Networks*. Geifman and El-Yaniv (2017), *Selective Classification*. Zaheer et al. (2017), *Deep Sets*; Lee et al. (2019), *Set Transformer*. Williams (1992), REINFORCE; Shao et al. (2024), GRPO. Hinton, Vinyals and Dean (2015), *Distillation*. Related decision models: Laya (Convai Innovations), TypeSafe Jev, Together Tev1.

**Data.** ARC, OpenBookQA, SciQ, CommonsenseQA, QASC, HellaSwag, WinoGrande, MMLU, BoolQ, GSM8K, AQuA-RAT, SNLI, MultiNLI, ANLI, SciTail, CLINC150, MASSIVE, Bitext, Banking77, AG News, Yelp, DAIR Emotion, SST-5, jailbreak-classification, Civil Comments, deepset prompt-injections, CodeXGLUE, MBPP, HumanEval, LocalLLaMA/typed-decisions.

## Architecture
- Identification: **NLP · Small Language Model (SLM)** (98% confidence)
- Source format: `safetensors` · Intended task: not declared
- `config.json`: synthesized from the anatomy (no source config.json); model_type omitted — no architecture name in the source (QA-F-126)
- Source license: not recorded (uploaded file)
- Lineage chain: 1 surgery (no prior attestation reachable)
- Post-surgery totals: 159,654,157 parameters ·
  168 tensors
- Compute estimate: 15.02439 GFLOPs (comparison
  metric, not a measurement)

## Provenance & operations
- Parents: model.safetensors
- Operations performed: load×1
- Weight merges recorded: 0
- Quantized tensors (F32→F16): 0

## Surgery Log (ordered)
1. **load** — model.safetensors (319.3 MB, safetensors)

## Validation
- Tissue imaging: not run
- Structural integrity is testable offline via the packaged
  `load_and_test.py`.

## Compliance note
The signed attestation + this card together document model composition,
modification history, and validation evidence — the record structure
technical-documentation obligations (e.g. EU AI Act Annex IV) ask for.
This is evidence, not legal advice.

---
*Operated with Model Surgeon — verify this package at https://surgeon.falcons.ai/verify*
*© 2026 FALCONS.AI — Model Surgeon record format. The model weights remain their owner's.*