opticsWolf/decider-2b-fp32-onnx

decider-2b (weightless ONNX, fp32 compute)

Weightless ONNX export of Mapika/decider-2b v12 (Qwen3.5-2B-Base, Apache-2.0): one forward pass, typed decisions with calibrated probabilities — no generation. Same bytes Ollaya serves as decider:2b, staged here self-contained.

• model.onnx (7.6 MB graph) + model.safetensors (3.76 GB BF16, upstream bytes vendored) + tokenizer.json + decision.json + calibration.json.
• License: Apache-2.0.
• Graph built algebraically (ollaya convert/ollaya_convert: every checkpoint tensor maps to an external BF16 reference + Cast to fp32), not torch-traced: the Gated-DeltaNet chunked rule rejects symbolic sequence dims, so dynamo export hangs. Provenance + hashes: manifest.json.

Contract (decider-slots-v1)

Inputs: input_ids int64 [rows, seq] (seq a multiple of 64, right-pad any id), slot_pos int64 [rows] (answer-slot positions). Output: label_logits float32 [rows, 255] — tied-LM-head rows for the 255 option labels; use the first k. Score questions run one isolated yes/no row per level (decision.json has the full layout).

Serve

import onnxruntime as ort
so = ort.SessionOptions()
so.enable_mem_reuse = False  # required on CUDA (see below)
sess = ort.InferenceSession("model.onnx", so, providers=["CUDAExecutionProvider"])

CUDA requires enable_mem_reuse = False: the CUDA memory planner otherwise corrupts the int64 shape values the chunk-scan Reshapes read (requested shape {8,16,0,64} — documented in Ollaya's runner). CPU keeps defaults. Temperatures come from calibration.json (choice 2.465 / score 1.0 / noul 0.05 — the 0.05 is deliberate, for JevBench's 0.2–0.8 band rule; calibrated yes/no needs 3.185, see the upstream card).

Measured (RTX 3090 + Ryzen 9 5950X)

Check	Result
Graph	opset 18, 7,976 nodes, zero non-standard ops, 0 unused tensors
torch↔ORT parity	max|Δp| 1.0e-4 CPU / 1.9e-4 CUDA (choice + noul + score, winners agree 3/3)
CPU	~1.5–2 s/5-row batch — slow-tier only
CUDA	373 ms/5-row batch (75 ms/row; Ollaya: 91 ms/5q on 5090)

Full bakeoff notes: see the embroider repo, docs/typed-decision-plan.md §9c (dynamo rejection, CUDA workaround, int8 status).

q8

decider-2b q8: 186 decoder linears to int8 MatMulNBits (block 32, fp32 scales); slot head (dynamic-B), embeddings, norms untouched (still served from model.safetensors). Gate: worst 0.0020, 0 flips, 10/10 golden agree (incl. 0.05-temp noul). Parity: worst 0.0020, 0 flips (10/10 golden agree).