Darwin-27B-RSI-Decision

A one-forward-pass decision engine from the Darwin family. It reads a piece of state and a set of typed questions, and returns a probability distribution over the options of every question. No tokens are generated.

It takes the POST /v1/systemone request shape: noul (yes or no), choice (one of N labels) and score (an ordered rubric).

Results

Decision Index (public suite, self-run)

Run with the public reproduction kit (apolinario/decision-index), http engine, every request answered.

Metric	Darwin-27B-RSI-Decision
Balanced skill ↑	68.77
Knowledge & Reasoning	54.00
Language	71.74
Retrieval	73.06
Tools	85.59
Arts	59.99

The board's official number comes from the operator-run private evaluation; the table above is our own run of the public suite.

How it works

• Backbone: FINAL-Bench/Darwin-27B-RSI, full-weight trained as a decision engine.
• Prompt attention: the prompt (state, question and option list) is read with bidirectional attention; the answer readout stays one forward pass.
• Readout: the options are listed in the prompt with short codes. A readout head maps the final hidden state to one score per option. A softmax gives the distribution, and a fitted temperature calibrates it.
• Training data: training and development splits of public decision benchmarks, plus synthetic decision data written for this model (multi-step reasoning stories, code-output questions, financial-entity sentiment, review aspect/sentiment pairs) and schema-variant dialogues built from the SGD and SGD-X training and development splits (CC-BY-SA-4.0). No test split of any benchmark was used for training. Synthetic rows were checked against the public Decision Index suite for 13-gram overlap.

Files

File	What it is
model-*.safetensors, model.safetensors.index.json, config.json	backbone weights (BF16)
readout.safetensors	decision readout head
decision_config.json	option codes, calibration temperature, prompt-attention mode and provenance
tokenizer.json, tokenizer_config.json, chat_template.jinja	tokenizer and prompt template
autojev/	inference code
decision_server.py	the /v1/systemone server used for the Decision Index run

Usage

This repo ships the inference code in autojev/, based on the autojev decision-model code (MIT, copyright notice in autojev/LICENSE), with bidirectional prompt attention.

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("FINAL-Bench/Darwin-27B-RSI-Decision")
sys.path.insert(0, path)
from autojev.model import DecisionModel

model = DecisionModel(checkpoint=path)
row = {
    "state": {"ticket": "Customer was charged twice for the same order."},
    "question": {
        "type": "choice",
        "instructions": "What should support do?",
        "criteria": {"refund": "Refund the duplicate charge.", "escalate": "Send to billing.", "close": "No action."},
    },
}
probabilities = model.predict([row])[0]  # one probability per option, in criteria order

/v1/systemone server

The Decision Index run used decision_server.py from this repo with its defaults (one forward pass per question, batch 8, max length 32768, nothing truncated):

pip install torch transformers fastapi uvicorn httpx pillow
huggingface-cli download FINAL-Bench/Darwin-27B-RSI-Decision --local-dir Darwin-27B-RSI-Decision
cd Darwin-27B-RSI-Decision && PORT=8090 python decision_server.py

Citation

@misc{darwin27brsidecision2026,
  title  = {Darwin-27B-RSI-Decision: a one-pass decision engine},
  author = {VIDRAFT and FINAL-Bench},
  year   = {2026},
  url    = {https://huggingface.co/FINAL-Bench/Darwin-27B-RSI-Decision}
}