Jeffan-9B-1211

A research model built on Qwen3.5-9B to answer structured decision questions. The input is a state (a document, a conversation, a list of tools, …) and one or more multiple-choice questions about it. The output is a probability for every option, read from a single forward pass; the model generates no text.

Status: first release.

Model

Base Qwen/Qwen3.5-9B (Apache 2.0)
Weights bf16, 8.95B parameters, text only (Qwen3_5ForCausalLM); same tokenizer and chat template as the base
Attention bidirectional in the full-attention layers: every token of the prompt sees every other token. The linear-attention (Gated DeltaNet) layers stay causal. The remote code below applies it; the stock causal mask gives a different model
Layer loop Blocks of middle layers (4–7, 8–11, 12–15) run P times per forward pass with a damped update (x ← x + (block(x) βˆ’ x) / P, fp32, no extra parameters). P is set at inference; default 2. See Layer loop passes.
Context prompts up to 65,536 tokens with the engine below (every prompt of the public suite fits)

Inference

The model reads one question at a time in its own prompt layout (a fixed system line; State:, Question: and Options: with the codes A, B, …; a closing line) and answers with its next-token distribution over the option codes. Nothing is generated. Results with other prompt formats may differ.

transformers

import torch
from transformers import AutoModel

# 2 loop passes by default (loop_passes=N at load to change)
model = AutoModel.from_pretrained("tattrongvu/Jeffan-9B-1211", trust_remote_code=True,
                                  dtype=torch.bfloat16, device_map="cuda")
state = "Ticket #1042: the parcel arrived with the screen cracked; the customer asks for a replacement."
question = "Which team should handle this ticket?"
options = ["Billing", "Shipping damage claims", "Technical support"]
print(model.decide(state, question, options))
# {'Billing': 0.0107, 'Shipping damage claims': 0.9620, 'Technical support': 0.0273}
model.set_loop_passes(3)        # 3 passes: kit sample 70.0 (vs 69.8), about 1.3x the latency
print(model.decide(state, question, options))
# {'Billing': 0.0144, 'Shipping damage claims': 0.9509, 'Technical support': 0.0346}
model.set_loop_passes(1)        # 1 pass (each layer once): kit sample 66.9, about 0.6x the latency
print(model.decide(state, question, options))
# {'Billing': 0.0012, 'Shipping damage claims': 0.9908, 'Technical support': 0.0081}
  • model.decide(state, question, options) returns the probability of every option, from one forward pass; nothing is generated.
  • state is text or any JSON-serialisable object; options is a list of up to 255 distinct texts, shown in that order; ordered_levels=True adds a line for options that are ordered levels (a score); tokenizer= reuses a loaded tokenizer.
  • Loop passes: 2 by default (loop_passes=N at load or model.set_loop_passes(N) later). 3 or 4 passes score about the same (slightly higher on the kit sample) and take longer; 1 pass is about 1.6Γ— faster but about 3 points lower on the kit sample. See Layer loop passes.
  • The remote code, modeling_jeffan.py (Apache 2.0), is the native Qwen3.5 model plus the bidirectional prompt mask, the layer loop and the readout; the weights load unchanged.
  • Without remote code: install the package in inference/ and use from jeffan_engine.hf import load, decide (the same code): model = load("tattrongvu/Jeffan-9B-1211"), then decide(model, state, question, options).
  • Verified on one NVIDIA B300 (bf16, transformers 5.17.0, PyTorch 2.14.1): on 300 held-out questions, decide builds exactly the prompt tokens of our evaluation engine and returns the same probabilities (300 of 300, largest difference 0.0), at 1 and at 2 passes.

Decision Index engine

inference/ has jeffan_engine, an engine for the Decision Index kit on transformers (any CUDA GPU with enough memory; Apache 2.0): every question of a request in its own prompt, the bidirectional mask and the layer loop above, up to 255 options, nothing truncated. Tested with PyTorch 2.14.1, transformers 5.17.0, flash-linear-attention 0.5.2 and causal-conv1d 1.7.0 (optional) on the kit at f8552a1, on NVIDIA B300.

pip install torch "transformers==5.17.0" "flash-linear-attention==0.5.2"
pip install causal-conv1d==1.7.0 --no-build-isolation   # optional
git clone https://github.com/apolinario/decision-index && git -C decision-index checkout f8552a1
pip install -e ./decision-index
hf download tattrongvu/Jeffan-9B-1211 --include "inference/*" --local-dir Jeffan-9B-1211
pip install ./Jeffan-9B-1211/inference
python -m decision_index pipeline --engine jeffan_engine:DecisionEngine \
    --option model=tattrongvu/Jeffan-9B-1211 --option max_len=65536 --option budget=65536 \
    --out runs/Jeffan-9B-1211 --compact

Details and declared limits: inference/README.md.

vLLM API server (/v1/systemone, /v1/decisions)

vLLM serves decision models on two endpoints: POST /v1/systemone (System One format: a state and a map of choice, noul and score questions) and POST /v1/decisions (the OpenAI Decisions API, also reachable with the openai SDK's client.decisions.create). Stock vLLM has no read strategy for this architecture and answers HTTP 501. The plugin in inference/vllm_decisions/ (jeffan_vdec, Apache 2.0, two Python files) makes the model's own prompt layout and readout the read strategy of both endpoints: every question in its own prompt, token for token the prompt of the Decision Index engine above (it loads the repo's inference/jeffan_engine/render.py), the codes A, B, … for up to 255 options, and the probabilities renormalised over the option codes. It changes nothing in the engine; there is nothing to install beyond vLLM: the plugin directory goes on PYTHONPATH, with JEFFAN_VDEC=1.

vLLM runs this model with 1 pass of the layer loop: it serves the plain layer stack (each layer once) and does not apply the loop in dm_config.json. That is 66.9 on the kit sample against 69.8 at the default of 2 passes (see Layer loop passes); every number in Evaluation, including the public index of 70.55, is at 2 passes with the transformers code or the Decision Index engine above.

--hf-overrides '{"is_causal": false}' is vLLM's switch for the bidirectional prompt: the full-attention layers read the whole prompt in both directions (the Gated DeltaNet layers stay causal). Every prompt must then be read whole in one step, so prefix caching and chunked prefill are off. On Blackwell GPUs use --attention-backend FLASH_ATTN (verified on the RTX PRO 6000 Blackwell Server Edition, where vLLM 0.33.0's TRITON_ATTN kernel for bidirectional attention needs more shared memory than the card has and the server fails at start). --max-logprobs 255 and --logprobs-mode processed_logprobs let the plugin read up to 255 option codes.

pip install "vllm==0.33.0"
hf download tattrongvu/Jeffan-9B-1211 --include "inference/*" --local-dir Jeffan-9B-1211
PYTHONPATH=$PWD/Jeffan-9B-1211/inference/vllm_decisions JEFFAN_VDEC=1 \
vllm serve tattrongvu/Jeffan-9B-1211 --max-model-len 65536 \
    --hf-overrides '{"is_causal": false}' --attention-backend FLASH_ATTN \
    --no-enable-prefix-caching --no-enable-chunked-prefill --max-num-seqs 128 \
    --max-logprobs 255 --logprobs-mode processed_logprobs

POST /v1/systemone, a choice, a yes / no (noul) and a score question about one state:

curl -s http://localhost:8000/v1/systemone -H "Content-Type: application/json" -d '{
  "model": "tattrongvu/Jeffan-9B-1211",
  "state": "Ticket #1042: the parcel arrived with the screen cracked; the customer asks for a replacement.",
  "questions": {
    "team": {"type": "choice", "instructions": "Which team should handle this ticket?",
             "criteria": {"Billing": null, "Shipping damage claims": null, "Technical support": null}},
    "replacement": {"type": "noul", "instructions": "Does the customer ask for a replacement?"},
    "urgency": {"type": "score", "instructions": "How urgent is this ticket?", "criteria": ["low", "medium", "high"]}
  }
}' | python3 -m json.tool
{
    "id": "decision-90358ee63e7a1c4f",
    "object": "structured_decision",
    "created": 1794588345,
    "model": "tattrongvu/Jeffan-9B-1211",
    "answers": {
        "team": {
            "type": "choice",
            "choice": "Shipping damage claims",
            "probabilities": {
                "Billing": 0.0010898826457712608,
                "Shipping damage claims": 0.9908569133433739,
                "Technical support": 0.008053204010854808
            },
            "confidence": 0.9495448801623981
        },
        "replacement": {
            "type": "noul",
            "noul": 0.9867771768866846,
            "probabilities": {
                "yes": 0.9867771768866846,
                "no": 0.01322282311331546
            },
            "confidence": 0.963022191925254
        },
        "urgency": {
            "type": "score",
            "score": 1.2527442837987297,
            "legend": {
                "0": "low",
                "1": "medium",
                "2": "high"
            },
            "probabilities": {
                "0": 0.14709131370463893,
                "1": 0.4530730887919924,
                "2": 0.39983559750336867
            },
            "confidence": 0.2234463557285315
        }
    },
    "usage": {
        "input_tokens": 356,
        "output_tokens": 3
    },
    "diagnostics": {
        "team": {
            "label_mass": 0.9583067619303581,
            "argmax_is_label": true
        },
        "replacement": {
            "label_mass": 0.9759266980247978,
            "argmax_is_label": true
        },
        "urgency": {
            "label_mass": 0.49317949191000526,
            "argmax_is_label": true
        }
    },
    "joint": null
}

POST /v1/decisions, the same questions in the OpenAI Decisions format (predicate for yes / no; answers come back in the order of the questions):

curl -s http://localhost:8000/v1/decisions -H "Content-Type: application/json" -d '{
  "model": "tattrongvu/Jeffan-9B-1211",
  "input": "Ticket #1042: the parcel arrived with the screen cracked; the customer asks for a replacement.",
  "questions": [
    {"type": "choice", "name": "team", "instructions": "Which team should handle this ticket?",
     "choices": [{"value": "Billing"}, {"value": "Shipping damage claims"}, {"value": "Technical support"}]},
    {"type": "predicate", "name": "replacement", "instructions": "Does the customer ask for a replacement?"},
    {"type": "score", "name": "urgency", "instructions": "How urgent is this ticket?",
     "levels": [{"label": "low"}, {"label": "medium"}, {"label": "high"}]}
  ]
}' | python3 -m json.tool
{
    "model": "tattrongvu/Jeffan-9B-1211",
    "answers": [
        {
            "type": "choice",
            "name": "team",
            "choice": "Shipping damage claims",
            "probabilities": [
                {
                    "value": "Billing",
                    "probability": 0.0010898826457712608
                },
                {
                    "value": "Shipping damage claims",
                    "probability": 0.9908569133433739
                },
                {
                    "value": "Technical support",
                    "probability": 0.008053204010854808
                }
            ],
            "confidence": 0.9495448801623981
        },
        {
            "type": "predicate",
            "name": "replacement",
            "probability": 0.9867771768866846
        },
        {
            "type": "score",
            "name": "urgency",
            "score": 1.2527442837987297,
            "probabilities": [
                {
                    "value": 0,
                    "label": "low",
                    "probability": 0.14709131370463893
                },
                {
                    "value": 1,
                    "label": "medium",
                    "probability": 0.4530730887919924
                },
                {
                    "value": 2,
                    "label": "high",
                    "probability": 0.39983559750336867
                }
            ],
            "confidence": 0.2234463557285315
        }
    ],
    "usage": {
        "input_tokens": 356,
        "input_tokens_details": {
            "cached_tokens": 0,
            "cache_write_tokens": 0
        },
        "output_tokens": 3,
        "output_tokens_details": {
            "reasoning_tokens": 0
        },
        "total_tokens": 359
    }
}

With the openai SDK (pip install openai, a client of the server above):

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
decision = client.decisions.create(
    model="tattrongvu/Jeffan-9B-1211",
    input="Ticket #1042: the parcel arrived with the screen cracked; the customer asks for a replacement.",
    questions=[
        {"type": "choice", "name": "team", "instructions": "Which team should handle this ticket?",
         "choices": [{"value": "Billing"}, {"value": "Shipping damage claims"}, {"value": "Technical support"}]},
        {"type": "predicate", "name": "replacement", "instructions": "Does the customer ask for a replacement?"},
        {"type": "score", "name": "urgency", "instructions": "How urgent is this ticket?",
         "levels": [{"label": "low"}, {"label": "medium"}, {"label": "high"}]},
    ],
)
for a in decision.answers:
    if a.type == "choice":
        print(a.name, a.choice, {p.value: round(p.probability, 4) for p in a.probabilities})
    elif a.type == "predicate":
        print(a.name, round(a.probability, 4))
    elif a.type == "score":
        print(a.name, round(a.score, 3), {p.label: round(p.probability, 4) for p in a.probabilities})
team Shipping damage claims {'Billing': 0.0012, 'Shipping damage claims': 0.9903, 'Technical support': 0.0086}
replacement 0.9859
urgency 1.253 {'low': 0.1471, 'medium': 0.4531, 'high': 0.3998}
  • A choice answer is the argmax and the probability of every option; a noul / predicate answer is P(true); a score is the expected 0-indexed level, with the probability of each level.
  • At most 64 questions per request on either endpoint (larger requests get HTTP 400: split them); up to 255 options per choice (stock vLLM: 26); 2–10 levels per score.
  • Instructions, option names and descriptions, and levels are written as text: send structured values as JSON text (json.dumps). The state of /v1/systemone may be text or JSON (written as JSON); /v1/decisions takes text input only (no images).
  • The prompt is fixed: a request-level instructions field or chat_template_kwargs gets HTTP 400. A noul may give its two sides a text ("criteria": {"true": "…", "false": "…"}); a predicate has no place for that, so on /v1/decisions send a choice over the JSON booleans false then true with descriptions, which is read as the same yes / no prompt.

Verified on one NVIDIA RTX PRO 6000 Blackwell Server Edition, with a fresh environment from PyPI (vllm==0.33.0, PyTorch 2.14.0+cu130, openai 3.50.0) and the commands above verbatim: the outputs shown are theirs. Probabilities can differ from run to run in the third or fourth decimal: the questions of a request are read as separate prompts and may be batched differently. On 300 held-out requests (630 questions), the answers choose the same option as the same model at 1 pass with the Decision Index engine's prompts sent to vLLM's completions endpoint on 628 of 630 questions through /v1/systemone and 629 of 630 through /v1/decisions (largest KL divergence 0.002); the plugin's prompts are token for token those of the Decision Index engine. Requests with 65 questions or 256 options get HTTP 400; 64 questions and 255 options are answered.

Evaluation: full public suite (Jev Decision Index 0.3)

All 140,178 public requests of the 0.3 edition, HLE included, answered by the Decision Index engine above (the kit's sequential runner, one request at a time, bf16, one NVIDIA B300 per shard) and scored with the kit's own scoring code: the same computation as the leaderboard's public score. Not an official leaderboard result: since Decision Index 0.4 the Full score comes from private tests that only the leaderboard can run, so this model has no Full score yet.

model public index Knowledge Language Retrieval Tools Arts leaderboard Full
Jeffan-9B-1211, our run 70.55 56.8 72.7 74.3 88.6 60.0 –
GLiDE-9B (Fastino), leaderboard 68.77 62.48
elephant-1 (baselabs, 9B), leaderboard 72.24 60.26
Cerno 9B v1.2, leaderboard 66.83 58.98
KnowLine 9B Gen1, leaderboard 65.67 58.44
StartLux-Decision v1.1 9B, leaderboard 69.98 58.07
Jeffan-4B-2010, leaderboard 64.85 52.21

Leaderboard values from its published data of 2026-11-06.

Latency per request on one NVIDIA B300 (bf16; the kit's sequential runner, one request at a time; a 700-request set built like the leaderboard's latency set, after 10 untimed warm-up requests):

setting median mean p95
2 passes (default) 74 ms 97 ms 205 ms
1 pass 45 ms 67 ms 153 ms
vLLM (1 pass) 29 ms 42 ms 87 ms
stock Qwen3.5-9B 45 ms 67 ms 155 ms

On the kit's fixed sample of the public suite (4,150 requests, HLE excluded), the sequential runner gives an index of 70.01.

Layer loop passes

The number of passes of the layer loop is set at inference (loop_passes / model.set_loop_passes(n) in transformers, --option loop_passes=N in the Decision Index engine). Fewer passes are faster but less accurate; 2 is the default and the setting of every other number on this card.

passes kit sample index (4,150 requests, HLE excluded; concurrent runner) median latency per request, ms (one B300)
1 66.9 47
2 (default) 69.8 77
3 70.0 103
4 71.0 121

This table comes from a separate run (the kit's concurrent runner for the index), so its 2-pass row differs slightly from the sequential numbers above.

Limitations

  • A first release; later versions may differ.
  • Quality on non-English inputs and on unfamiliar task types is lower.
  • Probabilities are not validated for any production use. Do not use it for high-stakes decisions without human review.
  • Not affiliated with any leaderboard.

License

Apache 2.0 (LICENSE), for the weights and the inference code in inference/. The base model, Qwen3.5-9B, is released by the Qwen team under Apache 2.0 too.

Downloads last month
214
Safetensors
Model size
9B params
Tensor type
BF16
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for tattrongvu/Jeffan-9B-1211

Finetuned
Qwen/Qwen3.5-9B
Finetuned
(1,416)
this model
Quantizations
3 models

Space using tattrongvu/Jeffan-9B-1211 1

Collection including tattrongvu/Jeffan-9B-1211