Upload README.md

README.md

@@ -0,0 +1,117 @@
+---
+license: apache-2.0
+language: [en]
+library_name: transformers
+tags: [decision-model, system-one, calibrated-decisions, multiple-choice, zero-shot-classification, falcondec]
+---
+# FalconDec v1.0.0 — Falcon Decision
+
+Single-pass, typed (`choice` / `noul` / `score`), calibrated closed-set decision model. Successor to
+[`Falconsai/proof_v2`](https://huggingface.co/Falconsai/proof_v2).
+
+* Built with FalconDec notebook V2 · backbone `jhu-clsp/ettin-encoder-150m` · mode `scratch` · preset `standard` · lineage: jhu-clsp/ettin-encoder-150m
+* Parameters: 159.7M · weights: fp16 305 MB, int8 153 MB
+* Layout: `[CLS] question [SEP] [MASK] opt1 … [MASK] optk [SEP] state [SEP]`; set-transformer option head;
+  temperature per (question type × option-count bucket).
+
+## Usage
+
+```python
+import importlib.util
+from huggingface_hub import snapshot_download
+path = snapshot_download("<repo>")            # or a local FalconDec-v1.0.0 directory
+spec = importlib.util.spec_from_file_location("falcondec_modeling", f"{path}/falcondec_modeling.py")
+fdm = importlib.util.module_from_spec(spec); spec.loader.exec_module(fdm)
+model, tok = fdm.load_falcondec(path)          # add "/compact-int8" for the int8 artefact
+out = fdm.decide(model, tok, state={"body": "I was charged twice, refund me or I cancel."}, questions={
+    "dept": {"type": "choice", "instructions": "Which team?", "criteria": {"billing": "payments", "tech": "bugs"}},
+    "churn": {"type": "noul", "instructions": "Does the user threaten to leave?"},
+    "urgency": {"type": "score", "instructions": "How urgent?", "criteria": ["low", "medium", "high"]}})
+```
+
+## Test results
+
+Overall: micro acc **0.725**, task-macro acc **0.725**, ECE **0.025**,
+NLL 0.652, Brier 0.358, AURC 0.097.
+
+| task | domain | heldout | n | chance | acc | proof_v2 (card) | ece |
+|---|---|---|---|---|---|---|---|
+| counsel/critique_quality | agentic | False | 201 | 0.333 | 0.557 |  | 0.094 |
+| counsel/step_has_error | agentic | False | 201 | 0.500 | 0.791 |  | 0.093 |
+| hotpotqa/comparison_yes_no | agentic | False | 17 | 0.500 | 0.824 |  | 0.185 |
+| hotpotqa/retrieve | agentic | False | 298 | 0.168 | 0.842 |  | 0.078 |
+| sst5/score | classification | True | 300 | 0.200 | 0.370 |  | 0.086 |
+| emotion/6way | classification | True | 300 | 0.167 | 0.480 |  | 0.049 |
+| yelp/score | classification | False | 300 | 0.200 | 0.640 |  | 0.069 |
+| ag_news/topic | classification | False | 300 | 0.250 | 0.847 |  | 0.051 |
+| devign/vulnerability | code | False | 300 | 0.500 | 0.553 | 0.542 | 0.026 |
+| mbpp/bugspot | code | False | 156 | 0.413 | 0.731 | 0.475 | 0.086 |
+| humaneval/completion | code | True | 119 | 0.394 | 0.756 | 0.575 | 0.152 |
+| bigclonebench/clone | code | False | 300 | 0.500 | 0.863 | 0.383 | 0.098 |
+| codexglue/func_name | code | False | 275 | 0.263 | 0.938 | 0.901 | 0.021 |
+| codexglue/doc_to_code | code | False | 300 | 0.274 | 0.973 | 0.961 | 0.022 |
+| codexglue/code_to_doc | code | False | 300 | 0.256 | 0.977 | 0.969 | 0.016 |
+| mbpp/solution | code | False | 300 | 0.250 | 0.987 | 0.992 | 0.014 |
+| codexglue/lang_id | code | False | 300 | 0.235 | 1.000 | 0.997 | 0.004 |
+| prompt_injections/detect | guardrails | True | 116 | 0.500 | 0.647 |  | 0.272 |
+| agentharm/refuse | guardrails | True | 416 | 0.500 | 0.654 |  | 0.178 |
+| civil_comments/toxic | guardrails | False | 300 | 0.500 | 0.807 |  | 0.051 |
+| jailbreak/detect | guardrails | False | 262 | 0.500 | 0.966 |  | 0.018 |
+| banking77/intent_77 | intents | True | 300 | 0.013 | 0.470 |  | 0.200 |
+| clinc150/intent | intents | False | 300 | 0.122 | 0.793 | 0.850 | 0.081 |
+| massive_en/intent | intents | False | 300 | 0.122 | 0.907 |  | 0.046 |
+| banking77/intent | intents | True | 300 | 0.317 | 0.923 | 0.883 | 0.037 |
+| policy/table_extreme_transfer | policy | False | 300 | 0.240 | 0.290 |  | 0.036 |
+| policy/invoice_overdue_transfer | policy | False | 300 | 0.500 | 0.407 |  | 0.364 |
+| policy/count_threshold_transfer | policy | False | 300 | 0.179 | 0.523 |  | 0.085 |
+| policy/invoice_total_transfer | policy | False | 300 | 0.500 | 0.523 |  | 0.020 |
+| policy/refund_approval_transfer | policy | False | 300 | 0.333 | 0.710 |  | 0.219 |
+| policy/sla_urgency_transfer | policy | False | 300 | 0.250 | 0.713 |  | 0.214 |
+| policy/free_shipping_transfer | policy | False | 300 | 0.500 | 0.850 |  | 0.067 |
+| policy/access_control_transfer | policy | False | 300 | 0.333 | 1.000 |  | 0.001 |
+| policy/return_window_transfer | policy | False | 300 | 0.333 | 1.000 |  | 0.019 |
+| aqua_rat/math | reasoning | False | 247 | 0.200 | 0.259 |  | 0.071 |
+| hellaswag/continuation | reasoning | False | 300 | 0.250 | 0.383 |  | 0.171 |
+| anli/nli | reasoning | False | 300 | 0.333 | 0.393 |  | 0.185 |
+| mmlu/mcq | reasoning | True | 300 | 0.250 | 0.393 |  | 0.094 |
+| arc_challenge/mcq | reasoning | True | 300 | 0.250 | 0.423 | 0.308 | 0.085 |
+| winogrande/blank | reasoning | False | 300 | 0.500 | 0.540 |  | 0.089 |
+| openbookqa/mcq | reasoning | False | 300 | 0.250 | 0.550 | 0.292 | 0.068 |
+| arc_easy/mcq | reasoning | True | 300 | 0.250 | 0.553 | 0.425 | 0.057 |
+| commonsense_qa/mcq | reasoning | False | 296 | 0.200 | 0.611 | 0.442 | 0.058 |
+| gsm8k/math | reasoning | False | 300 | 0.250 | 0.637 | 0.275 | 0.050 |
+| boolq/yes_no | reasoning | False | 300 | 0.500 | 0.783 | 0.717 | 0.071 |
+| mnli/claim | reasoning | False | 300 | 0.333 | 0.807 | 0.492 | 0.072 |
+| snli/nli | reasoning | False | 594 | 0.333 | 0.837 |  | 0.035 |
+| sciq/mcq | reasoning | False | 300 | 0.250 | 0.950 | 0.692 | 0.023 |
+| scitail/support | reasoning | False | 300 | 0.500 | 0.957 |  | 0.030 |
+| snli/contradicts | reasoning | False | 300 | 0.333 | 0.967 | 0.892 | 0.035 |
+| snli/must_be_true | reasoning | False | 300 | 0.333 | 0.970 | 0.908 | 0.029 |
+| qasc/mcq | reasoning | False | 300 | 0.125 | 0.983 |  | 0.007 |
+| bitext/category | support | False | 300 | 0.172 | 0.993 |  | 0.008 |
+| bitext/route | support | False | 300 | 0.200 | 1.000 | 0.958 | 0.002 |
+| typed_decisions/invoice_processing | workflows | False | 500 | 0.350 | 0.628 |  | 0.101 |
+| typed_decisions/agent_trace_observability | workflows | False | 500 | 0.300 | 0.696 |  | 0.106 |
+| typed_decisions/security_incidents | workflows | False | 500 | 0.340 | 0.712 |  | 0.140 |
+| typed_decisions/customer_service | workflows | False | 500 | 0.280 | 0.720 |  | 0.099 |
+
+
+## Training
+
+Strictly proper objective (log + 0.5·spherical + 1.0·RPS for ordinal), soft teacher targets where available,
+task-balanced sampling (α = 0.5), NOTA augmentation (8%), per-epoch option reshuffling,
+AdamW + LLRD 0.9, EMA 0.999, 2 epoch(s), RLCD stage off/reverted.
+Training time 46.1 min on NVIDIA GeForce RTX 5090 Laptop GPU.
+
+## Limitations
+
+* Closed world: the model ranks the options you give it. Add "None of the above" when that is a valid answer (it was trained with it).
+* Multi-step arithmetic (GSM8K / AQuA) is a System-2 task; route low-confidence answers to an LLM (`defer` flag, threshold 0.7).
+* English only; code judgement (vulnerabilities, clones) is still weak; check calibration on your own traffic.
+
+## References
+
+Warner et al. 2024 (ModernBERT) · Weller et al. 2025 (Ettin) · Gneiting & Raftery 2007 (strictly proper scoring rules) ·
+Guo et al. 2017 (temperature scaling) · Geifman & El-Yaniv 2017 (selective classification) · Shao et al. 2024 (GRPO) ·
+Williams 1992 (REINFORCE) · Zaheer et al. 2017 / Lee et al. 2019 (Deep Sets / Set Transformer) · Hinton et al. 2015 (distillation) ·
+Laya (convaiinnovations) · TypeSafe Jev · Together Tev1.