---
license: apache-2.0
base_model: distilroberta-base
tags: [system-one, structured-output, calibrated, classification, parallel-decoding]
---
# Alpha (System-One-style typed decision model)

Alpha takes unstructured / structured state plus **typed questions with declared options** and returns calibrated probabilities for
**all questions in one parallel forward pass**. It cannot generate strings and cannot answer outside the declared options
(masked softmax -> schema violations are impossible by construction). Option embeddings are independent of the state, so a schema can be
compiled once and cached; cardinality is limited only by memory.

Architecture: `distilroberta-base` encoder (state + question slots with `<mask>` readout tokens) -> 2-layer **Slot Reasoner** (set transformer, squared-ReLU,
zero-init residuals) -> biaffine/cosine scorer against option embeddings with per-question learned temperature + post-hoc temperature (T=1.04).

Training: 6000 steps x 32 workflow-style examples (1-4 questions over named fields, mixed from 10 public datasets with paraphrased
questions/option wordings), proper-scoring-rule loss (NLL + Brier + ordinal term), EMA weights, functional-memory anchor, discovery controller.
Held-out families (never trained on): cola, dbpedia, offensive.

## Results (test splits; see results.json, alpha_report.png)
| family | acc (seen wording) | acc (unseen wording) | ECE |
|---|---|---|---|
| ag_news | 0.926 | 0.924 | 0.032 |
| banking77 | 0.902 | 0.890 | 0.036 |
| emotion | 0.922 | 0.894 | 0.030 |
| imdb | 0.886 | 0.892 | 0.027 |
| mrpc | 0.822 | 0.822 | 0.042 |
| news20 | 0.674 | 0.678 | 0.068 |
| sms | 0.993 | 0.346 | 0.004 |
| snli | 0.772 | 0.538 | 0.046 |
| sst2 | 0.912 | 0.910 | 0.034 |
| yelp | 0.572 | 0.522 | 0.072 |

Workflow eval (4 questions per query): per-question acc 0.825, all-correct 0.437, joint-confidence ECE 0.101.

Held-out task families (unseen schema wording): zero-shot / after 10 / after 40 gradient steps
| family | zero-shot | +10 steps | +40 steps |
|---|---|---|---|
| cola | 0.374 | 0.658 | 0.686 |
| dbpedia | 0.312 | 0.586 | 0.952 |
| offensive | 0.528 | 0.632 | 0.694 |

Speed on Tesla T4: p50 11.1 ms, p95 13.5 ms for a 3-question query (local, no network); 354 workflows/s at batch 64.
Qwen2.5-0.5B-Instruct (generative, same 120 questions): acc 0.583, p50 latency 82 ms vs Alpha acc 0.875, p50 11.1 ms.

Framework diagnostics: DMR=1.08; FRR by perturbation: sigma=0.5: FRR 4250.0 sigma=1: FRR 4250.0 sigma=2: FRR 4250.0 sigma=4: FRR 4250.0 sigma=8: FRR 14.2* (* = not re-reached within budget); MGR by checkpoint in results.json.

## Honest limits
* This is a small research model trained on public classification datasets. **It is not a replacement for, and has not been compared against, TypeSafe's Jev**
  (a closed API; its claims were not verified here). Do not read the latency comparison as a Jev comparison.
* Narrow task coverage (sentiment/topic/intent/NLI/paraphrase/spam/rating); free-form reasoning is out of scope by design.
* Calibration is measured in-distribution; shift will degrade it.

## Usage
```python
from huggingface_hub import snapshot_download; import sys
p = snapshot_download("SofiTesfay2010/Alpha"); sys.path.insert(0, p)
from alpha_s1 import Alpha
m = Alpha.from_pretrained(p, device="cuda").eval()
print(m.predict({"ticket": "I was charged twice, refund me now."},
      [{"id": "refund", "question": "Does the customer want a refund?", "options": ["no", "yes"]}]))
```