decider-31b v2: text and image decisions, ECE_bw 0.018; v1 under tag v1

53e0a1b verified 1 day ago

12 kB

---
license: apache-2.0
base_model: google/gemma-4-31B-it
language: [en]
pipeline_tag: image-text-to-text
tags: [decision-model, calibrated, structured-output, system-one, one-pass, nvfp4, vision]
---

# decider-31b: typed decisions with calibrated probabilities on Gemma-4-31B, from text and images, one pass

decider-31b answers typed decisions with a probability for every option. It reads each question in one forward pass and
generates no text. From v2 the state may include images.

- **Input:** a state and one or more typed questions (Choice, Noul yes/no, Score), each with an explicit option list, in the
  System One wire format (`POST /v1/systemone`). Images are sent as Jev-style `assets` and referenced from the state.
- **Output:** a probability distribution over the options of every question. Nothing needs parsing.
- **Weights:** [google/gemma-4-31B-it](https://huggingface.co/google/gemma-4-31B-it) with a fine-tune merged into the
  attention and MLP projections, quantized to NVFP4 (MLP weights and activations in NVFP4, attention projections and the vision
  tower in bf16, FP8 KV cache). About 31 GB.
- **Code:** `decider.serve_vllm` in [decider-ai](https://github.com/Mapika/decider) 1.10.0. It reads this repository's
  `decider_config.json`, which holds the configuration measured below and switches image input on.
- **No thinking.** This model never generates a reasoning trace. Every request is answered in one or two forward passes.
- **Versions:** this repository holds v2 (2026-10-20). v1, the text-only model of 2026-10-07, stays available under the Hub tag
  `v1`.

## Changes from v1

v2 is v1 trained further on image decisions, with a part of v1's text data replayed. Measured on the same suites with the same
server:

| | v1 | v2 |
|---|---|---|
| Decision Index 0.3 public, all 140,178 requests | **64.23** | 63.90 |
| Calibration on the index, ECE_bw | 0.035 | **0.018** |
| Image questions, mean accuracy over 11 sets | 0.843 | **0.882** |
| Visual7W / VSR (never trained on) | 0.877 / 0.843 | 0.877 / 0.853 |
- The index score is 0.33 lower. The loss is in Retrieval & Classification (65.53 to 62.47); Knowledge & Reasoning and Language
  Understanding are higher (50.35 to 50.87, 66.60 to 67.32).
- Calibration on the index is better: ECE_bw 0.018 against 0.035.
- The image gain is on the kinds of image questions it was trained on (for example Hateful Memes 0.72 to 0.83, RAVEN 0.25 to
  0.47). On the two image sets it never saw, it is level.
- Use v1 (`revision="v1"`) if you need text only and the last third of a point on the index.

## Results

Decision Index 0.3, public suite, all 140,178 requests (kit `--edition 0.3`), run by us on 2026-10-19 with our development
server (the same readout as `decider.serve_vllm`; on v1 the two gave 64.23 and 64.27). The board's full score also includes
private tests that only the maintainers run; this is the public part only.

| | decider-31b v2 | v1 |
|---|---|---|
| Public index (chance-corrected) | **63.90** | 64.23 |
| Public raw | 73.14 | 73.24 |
| Coverage | 1.0 | 1.0 |
| Calibration, ECE_bw | **0.018** | 0.035 |
| Area | v2 | v1 |
|---|---|---|
| Knowledge & Reasoning | 50.87 | 50.35 |
| Language Understanding | 67.32 | 66.60 |
| Retrieval & Classification | 62.47 | 65.53 |
| Tools & Automation | 84.18 | 84.11 |
| Arts & Human Taste | 54.57 | 55.04 |
Image questions: the eval splits of 11 sets from [the_cauldron](https://huggingface.co/datasets/HuggingFaceM4/the_cauldron),
300 items each, read the same way as text questions:
| | stock Gemma-4-31B-it, our readout | v1 | v2 |
|---|---|---|---|
| mean of 11 sets | 0.825 | 0.843 | **0.882** |
| Visual7W (not trained) | 0.883 | 0.877 | 0.877 |
| VSR (not trained) | 0.820 | 0.843 | 0.853 |
Through `decider.serve_vllm` from decider-ai 1.10.0, with the image sent as a Jev-style asset, v2 gives 0.893 on Visual7W and 0.840 on VSR on the same
items, and 61.69 on our 4,500-request index sample (development server 61.78).

Notes on the numbers:

- The run was not verified by the Decision Index maintainers.
- ECE_bw is the benchmark-weighted expected calibration error over the 214,304 scored fields whose rows are in the suite's
  `selected-rows` file.
- The temperatures were fitted on our own rows only. No Decision Index row and no image evaluation item was used to fit or
  choose them.
- Training used the public train splits of several benchmarks that the index also tests (see Training data). No Decision
  Index test row was used: every training row was compared with all suite rows and dropped on a match.
## Latency
Not measured again for v2. The text path is the same model size, quantization and readout as v1, which measured median 39 ms,
mean 116 ms, p80 203 ms and p95 531 ms on one B300 (one request at a time, 400 Decision Index requests). The share of
questions read a second time can differ from v1's. Requests with images were not timed.
## Usage
vLLM 0.29.0 pins its own torch, so install in a separate environment:
```bash
python -m venv decider-vllm && . decider-vllm/bin/activate
pip install vllm==0.29.0 fastapi "uvicorn[standard]" jinja2 huggingface_hub pillow
pip install --no-deps "decider-ai>=1.10.0"
DECIDER_MODEL=Mapika/decider-31b uvicorn decider.serve_vllm:app --host 127.0.0.1 --port 8000
```
It needs one GPU with NVFP4 support (Blackwell: B200/B300, RTX PRO 6000, RTX 50 series with enough memory).
A text request:
```python
import httpx
r = httpx.post("http://127.0.0.1:8000/v1/systemone", timeout=60, json={
    "state": {"ticket": "Order #1182 arrived broken. Customer asks for a refund; order is 40 days old; policy: 30 days."},
    "questions": {"refund": {"type": "noul", "instructions": "Is the refund allowed under the policy?",
                             "criteria": {"true": "allowed", "false": "not allowed"}}}})
print(r.json()["answers"]["refund"])
```
A request with an image. The asset's key is the reference; every place the reference appears in the state is replaced by the
image:

```python
import base64, httpx
img = base64.b64encode(open("shelf.jpg", "rb").read()).decode()
r = httpx.post("http://127.0.0.1:8000/v1/systemone", timeout=60, json={
    "state": {"photo": "shelf-photo", "note": "Restock when fewer than 3 boxes are left."},
    "assets": {"shelf-photo": {"type": "image", "format": "jpg", "data_base64": img}},
    "questions": {"restock": {"type": "noul", "instructions": "Does the shelf in `photo` need restocking?",
                              "criteria": {"true": "restock", "false": "enough boxes"}}}})
```

Images are PNG or JPEG. A reference must appear in the state; references in instructions or criteria are not supported yet, and
an asset that the state never names is rejected (HTTP 422). `DECIDER_IMAGES=0` turns image input off and starts vLLM with the
language model only.

## How a request is answered

1. **First reading.** Every question is rendered in the model's chat template with thinking off, and the log-probabilities of
   the option letters are read at the answer slot. A softmax at T1(n) = 2.064 − 0.053 ln n (n = number of options) gives the
   probabilities. Each image in the state is one placeholder, which vLLM expands to the image's soft tokens (at most 280).
2. **Second reading, only when unsure.** A question whose top probability is below 0.7 is read again with its options in
   reversed order. The two log-softmaxes are averaged and a softmax at T2(n) = 1.904 − 0.022 ln n gives the final probabilities.
3. Questions above 0.7 keep the first reading.

The thresholds and temperatures are in `decider_config.json` (`temperature_by_options`, `second_reading`, `images`).
`DECIDER_SECOND_READING_BELOW=0` turns the second reading off. The config also selects vLLM's Triton attention backend
(`vllm_attention`): with this checkpoint's FP8 KV cache, the FlashInfer backend returned NaN in batched readouts in our tests.

## Training data

v2 starts from v1 and adds a LoRA stage (rank 32, learning rate 2e-5, 421 steps, 12.3M tokens, 2 hours on one B300) on:

- **Image decisions:** 18,129 rows from the train splits of seven the_cauldron subsets: AI2D 1,939, A-OKVQA 3,000, Hateful
  Memes 3,000, IconQA 3,000, RAVEN 3,000, ScienceQA 3,000, TQA 1,190. Yes/no questions are trained as Noul questions, the
  others as Choice questions. Visual7W and VSR were not used, so they measure image questions the model never saw.
- **Text replay:** 5,000 rows drawn from v1's training data.
v1's training data, unchanged:
- rows from public train splits of tasks where our previous entry was weak, rendered in the Decision Index request format, and
  decontaminated against every Decision Index 0.3 suite row: CLINC150 (train), iSarcasmEval (task A, English), VAST (train),
  ACOS (laptop and restaurant, train), POP909-CL (songs not used by the index), GSM8K (train), NLI4CT (train);
- generated smart-home requests from our own generator;
- from our earlier decider models: generated state-tracking decisions and decision families written by our own code, document
  questions written by Qwen3.6-27B, and labelled rows from public datasets (MMLU, ARC, CommonsenseQA, BoolQ, MNLI, SNLI,
  BANKING77, RACE, OpenBookQA, LogiQA 2.0, MedQA, WinoGrande).
Not used: Decision Index test rows, JevBench or ImageJevBench items.
## Limitations
- **One pass only.** Tasks that need several steps of reasoning (GPQA, ChessBench, HLE on the index) are not solved by reading
  harder.
- **Images:** the gain is on the kinds of image questions trained (diagrams, science figures, icons, memes, puzzles). On image
  question types it never saw, it reads about as well as v1 and the stock model. Box answers (Jev's `box` type) are not
  supported. We have not measured it on ImageJevBench.
- **Retrieval & Classification** is 3 points lower than v1 on the index.
- **Calibration:** check it on your own data before relying on the probabilities.
- **Hardware:** NVFP4 needs a Blackwell GPU. We measured only on B300.
- **vLLM version:** the server patches a vLLM 0.29 internal and refuses other versions.
- **Independent questions only:** `/decide` and the schema cache of `decider.serve` are not served.
## License
The weights are released under Apache 2.0, as the base model. Training sources and their terms:
- Image questions, from the train splits of [the_cauldron](https://huggingface.co/datasets/HuggingFaceM4/the_cauldron) subsets
  (the_cauldron licenses only its own prompts, CC BY 4.0; each source keeps its own terms):
  - AI2D (Kembhavi et al., 2016): CC BY-SA 4.0.
  - A-OKVQA (Schwenk et al., 2022): Apache 2.0 (repository); images from COCO.
  - IconQA (Lu et al., 2021): CC BY-NC-SA 4.0, non-commercial.
  - ScienceQA (Lu et al., 2022): CC BY-NC-SA 4.0, non-commercial research use.
  - TQA (Kembhavi et al., 2017): CC BY-NC 3.0, non-commercial.
  - RAVEN (Zhang et al., 2019): no data licence stated (the code is GPL-3.0).
  - Hateful Memes (Kiela et al., 2020): Facebook's Hateful Memes dataset licence. It permits use of the data to research and
    develop models designed to detect manipulated media, and states that models derived from it may be used for academic and
    commercial purposes. This is a research project; check the licence for your own use.
- CLINC150: CC BY 3.0 (Larson et al., 2019). Credit: "An Evaluation Dataset for Intent Classification and Out-of-Scope
  Prediction", clinc/oos-eval.
- iSarcasmEval: MIT (Abu Farha et al., 2022). The texts are tweets.
- POP909-CL: MIT; built on POP909 (Wang et al., 2020), MIT.
- VAST (Allaway and McKeown, 2020) and ACOS (Cai et al., 2021): no licence stated by the authors.
- GSM8K (Cobbe et al., 2021): MIT.
- NLI4CT (SemEval-2024 Task 2, Jullien et al.): no licence file in the task repository; released for the shared task.
- RACE: non-commercial research use. It is a small part of the replayed data (250 rows).
- ARC, SNLI, LogiQA 2.0 (copy dependent): CC BY-SA 4.0; BoolQ: CC BY-SA 3.0; BANKING77, MedQA: CC BY 4.0; MMLU,
  CommonsenseQA: MIT; MNLI: mixed per genre.

	---
	license: apache-2.0
	base_model: google/gemma-4-31B-it
	language: [en]
	pipeline_tag: image-text-to-text
	tags: [decision-model, calibrated, structured-output, system-one, one-pass, nvfp4, vision]
	---


	# decider-31b: typed decisions with calibrated probabilities on Gemma-4-31B, from text and images, one pass

	decider-31b answers typed decisions with a probability for every option. It reads each question in one forward pass and
	generates no text. From v2 the state may include images.

	- Input: a state and one or more typed questions (Choice, Noul yes/no, Score), each with an explicit option list, in the
	System One wire format (`POST /v1/systemone`). Images are sent as Jev-style `assets` and referenced from the state.
	- Output: a probability distribution over the options of every question. Nothing needs parsing.
	- Weights: [google/gemma-4-31B-it](https://huggingface.co/google/gemma-4-31B-it) with a fine-tune merged into the
	attention and MLP projections, quantized to NVFP4 (MLP weights and activations in NVFP4, attention projections and the vision
	tower in bf16, FP8 KV cache). About 31 GB.
	- Code: `decider.serve_vllm` in [decider-ai](https://github.com/Mapika/decider) 1.10.0. It reads this repository's
	`decider_config.json`, which holds the configuration measured below and switches image input on.
	- No thinking. This model never generates a reasoning trace. Every request is answered in one or two forward passes.
	- Versions: this repository holds v2 (2026-10-20). v1, the text-only model of 2026-10-07, stays available under the Hub tag
	`v1`.

	## Changes from v1

	v2 is v1 trained further on image decisions, with a part of v1's text data replayed. Measured on the same suites with the same
	server:

	\| \| v1 \| v2 \|
	\|---\|---\|---\|
	\| Decision Index 0.3 public, all 140,178 requests \| 64.23 \| 63.90 \|
	\| Calibration on the index, ECE_bw \| 0.035 \| 0.018 \|
	\| Image questions, mean accuracy over 11 sets \| 0.843 \| 0.882 \|
	\| Visual7W / VSR (never trained on) \| 0.877 / 0.843 \| 0.877 / 0.853 \|

	- The index score is 0.33 lower. The loss is in Retrieval & Classification (65.53 to 62.47); Knowledge & Reasoning and Language
	Understanding are higher (50.35 to 50.87, 66.60 to 67.32).
	- Calibration on the index is better: ECE_bw 0.018 against 0.035.
	- The image gain is on the kinds of image questions it was trained on (for example Hateful Memes 0.72 to 0.83, RAVEN 0.25 to
	0.47). On the two image sets it never saw, it is level.
	- Use v1 (`revision="v1"`) if you need text only and the last third of a point on the index.

	## Results

	Decision Index 0.3, public suite, all 140,178 requests (kit `--edition 0.3`), run by us on 2026-10-19 with our development
	server (the same readout as `decider.serve_vllm`; on v1 the two gave 64.23 and 64.27). The board's full score also includes
	private tests that only the maintainers run; this is the public part only.

	\| \| decider-31b v2 \| v1 \|
	\|---\|---\|---\|
	\| Public index (chance-corrected) \| 63.90 \| 64.23 \|
	\| Public raw \| 73.14 \| 73.24 \|
	\| Coverage \| 1.0 \| 1.0 \|
	\| Calibration, ECE_bw \| 0.018 \| 0.035 \|

	\| Area \| v2 \| v1 \|
	\|---\|---\|---\|
	\| Knowledge & Reasoning \| 50.87 \| 50.35 \|
	\| Language Understanding \| 67.32 \| 66.60 \|
	\| Retrieval & Classification \| 62.47 \| 65.53 \|
	\| Tools & Automation \| 84.18 \| 84.11 \|
	\| Arts & Human Taste \| 54.57 \| 55.04 \|

	Image questions: the eval splits of 11 sets from [the_cauldron](https://huggingface.co/datasets/HuggingFaceM4/the_cauldron),
	300 items each, read the same way as text questions:

	\| \| stock Gemma-4-31B-it, our readout \| v1 \| v2 \|
	\|---\|---\|---\|---\|
	\| mean of 11 sets \| 0.825 \| 0.843 \| 0.882 \|
	\| Visual7W (not trained) \| 0.883 \| 0.877 \| 0.877 \|
	\| VSR (not trained) \| 0.820 \| 0.843 \| 0.853 \|

	Through `decider.serve_vllm` from decider-ai 1.10.0, with the image sent as a Jev-style asset, v2 gives 0.893 on Visual7W and 0.840 on VSR on the same
	items, and 61.69 on our 4,500-request index sample (development server 61.78).

	Notes on the numbers:

	- The run was not verified by the Decision Index maintainers.
	- ECE_bw is the benchmark-weighted expected calibration error over the 214,304 scored fields whose rows are in the suite's
	`selected-rows` file.
	- The temperatures were fitted on our own rows only. No Decision Index row and no image evaluation item was used to fit or
	choose them.
	- Training used the public train splits of several benchmarks that the index also tests (see Training data). No Decision
	Index test row was used: every training row was compared with all suite rows and dropped on a match.

	## Latency

	Not measured again for v2. The text path is the same model size, quantization and readout as v1, which measured median 39 ms,
	mean 116 ms, p80 203 ms and p95 531 ms on one B300 (one request at a time, 400 Decision Index requests). The share of
	questions read a second time can differ from v1's. Requests with images were not timed.

	## Usage

	vLLM 0.29.0 pins its own torch, so install in a separate environment:

	```bash
	python -m venv decider-vllm && . decider-vllm/bin/activate
	pip install vllm==0.29.0 fastapi "uvicorn[standard]" jinja2 huggingface_hub pillow
	pip install --no-deps "decider-ai>=1.10.0"
	DECIDER_MODEL=Mapika/decider-31b uvicorn decider.serve_vllm:app --host 127.0.0.1 --port 8000
	```

	It needs one GPU with NVFP4 support (Blackwell: B200/B300, RTX PRO 6000, RTX 50 series with enough memory).

	A text request:

	```python
	import httpx
	r = httpx.post("http://127.0.0.1:8000/v1/systemone", timeout=60, json={
	"state": {"ticket": "Order #1182 arrived broken. Customer asks for a refund; order is 40 days old; policy: 30 days."},
	"questions": {"refund": {"type": "noul", "instructions": "Is the refund allowed under the policy?",
	"criteria": {"true": "allowed", "false": "not allowed"}}}})
	print(r.json()["answers"]["refund"])
	```

	A request with an image. The asset's key is the reference; every place the reference appears in the state is replaced by the
	image:

	```python
	import base64, httpx
	img = base64.b64encode(open("shelf.jpg", "rb").read()).decode()
	r = httpx.post("http://127.0.0.1:8000/v1/systemone", timeout=60, json={
	"state": {"photo": "shelf-photo", "note": "Restock when fewer than 3 boxes are left."},
	"assets": {"shelf-photo": {"type": "image", "format": "jpg", "data_base64": img}},
	"questions": {"restock": {"type": "noul", "instructions": "Does the shelf in `photo` need restocking?",
	"criteria": {"true": "restock", "false": "enough boxes"}}}})
	```

	Images are PNG or JPEG. A reference must appear in the state; references in instructions or criteria are not supported yet, and
	an asset that the state never names is rejected (HTTP 422). `DECIDER_IMAGES=0` turns image input off and starts vLLM with the
	language model only.

	## How a request is answered

	1. First reading. Every question is rendered in the model's chat template with thinking off, and the log-probabilities of
	the option letters are read at the answer slot. A softmax at T1(n) = 2.064 − 0.053 ln n (n = number of options) gives the
	probabilities. Each image in the state is one placeholder, which vLLM expands to the image's soft tokens (at most 280).
	2. Second reading, only when unsure. A question whose top probability is below 0.7 is read again with its options in
	reversed order. The two log-softmaxes are averaged and a softmax at T2(n) = 1.904 − 0.022 ln n gives the final probabilities.
	3. Questions above 0.7 keep the first reading.

	The thresholds and temperatures are in `decider_config.json` (`temperature_by_options`, `second_reading`, `images`).
	`DECIDER_SECOND_READING_BELOW=0` turns the second reading off. The config also selects vLLM's Triton attention backend
	(`vllm_attention`): with this checkpoint's FP8 KV cache, the FlashInfer backend returned NaN in batched readouts in our tests.

	## Training data

	v2 starts from v1 and adds a LoRA stage (rank 32, learning rate 2e-5, 421 steps, 12.3M tokens, 2 hours on one B300) on:

	- Image decisions: 18,129 rows from the train splits of seven the_cauldron subsets: AI2D 1,939, A-OKVQA 3,000, Hateful
	Memes 3,000, IconQA 3,000, RAVEN 3,000, ScienceQA 3,000, TQA 1,190. Yes/no questions are trained as Noul questions, the
	others as Choice questions. Visual7W and VSR were not used, so they measure image questions the model never saw.
	- Text replay: 5,000 rows drawn from v1's training data.

	v1's training data, unchanged:

	- rows from public train splits of tasks where our previous entry was weak, rendered in the Decision Index request format, and
	decontaminated against every Decision Index 0.3 suite row: CLINC150 (train), iSarcasmEval (task A, English), VAST (train),
	ACOS (laptop and restaurant, train), POP909-CL (songs not used by the index), GSM8K (train), NLI4CT (train);
	- generated smart-home requests from our own generator;
	- from our earlier decider models: generated state-tracking decisions and decision families written by our own code, document
	questions written by Qwen3.6-27B, and labelled rows from public datasets (MMLU, ARC, CommonsenseQA, BoolQ, MNLI, SNLI,
	BANKING77, RACE, OpenBookQA, LogiQA 2.0, MedQA, WinoGrande).

	Not used: Decision Index test rows, JevBench or ImageJevBench items.

	## Limitations

	- One pass only. Tasks that need several steps of reasoning (GPQA, ChessBench, HLE on the index) are not solved by reading
	harder.
	- Images: the gain is on the kinds of image questions trained (diagrams, science figures, icons, memes, puzzles). On image
	question types it never saw, it reads about as well as v1 and the stock model. Box answers (Jev's `box` type) are not
	supported. We have not measured it on ImageJevBench.
	- Retrieval & Classification is 3 points lower than v1 on the index.
	- Calibration: check it on your own data before relying on the probabilities.
	- Hardware: NVFP4 needs a Blackwell GPU. We measured only on B300.
	- vLLM version: the server patches a vLLM 0.29 internal and refuses other versions.
	- Independent questions only: `/decide` and the schema cache of `decider.serve` are not served.

	## License

	The weights are released under Apache 2.0, as the base model. Training sources and their terms:

	- Image questions, from the train splits of [the_cauldron](https://huggingface.co/datasets/HuggingFaceM4/the_cauldron) subsets
	(the_cauldron licenses only its own prompts, CC BY 4.0; each source keeps its own terms):
	- AI2D (Kembhavi et al., 2016): CC BY-SA 4.0.
	- A-OKVQA (Schwenk et al., 2022): Apache 2.0 (repository); images from COCO.
	- IconQA (Lu et al., 2021): CC BY-NC-SA 4.0, non-commercial.
	- ScienceQA (Lu et al., 2022): CC BY-NC-SA 4.0, non-commercial research use.
	- TQA (Kembhavi et al., 2017): CC BY-NC 3.0, non-commercial.
	- RAVEN (Zhang et al., 2019): no data licence stated (the code is GPL-3.0).
	- Hateful Memes (Kiela et al., 2020): Facebook's Hateful Memes dataset licence. It permits use of the data to research and
	develop models designed to detect manipulated media, and states that models derived from it may be used for academic and
	commercial purposes. This is a research project; check the licence for your own use.

	- CLINC150: CC BY 3.0 (Larson et al., 2019). Credit: "An Evaluation Dataset for Intent Classification and Out-of-Scope
	Prediction", clinc/oos-eval.
	- iSarcasmEval: MIT (Abu Farha et al., 2022). The texts are tweets.
	- POP909-CL: MIT; built on POP909 (Wang et al., 2020), MIT.
	- VAST (Allaway and McKeown, 2020) and ACOS (Cai et al., 2021): no licence stated by the authors.
	- GSM8K (Cobbe et al., 2021): MIT.
	- NLI4CT (SemEval-2024 Task 2, Jullien et al.): no licence file in the task repository; released for the shared task.
	- RACE: non-commercial research use. It is a small part of the replayed data (250 rows).
	- ARC, SNLI, LogiQA 2.0 (copy dependent): CC BY-SA 4.0; BoolQ: CC BY-SA 3.0; BANKING77, MedQA: CC BY 4.0; MMLU,
	CommonsenseQA: MIT; MNLI: mixed per genre.