---
license: apache-2.0
library_name: safetensors
tags:
  - adaptive-inference
  - routing
  - decision-model
base_model:
  - Qwen/Qwen3.5-4B
  - Qwen/Qwen3.8-27B
---

# HDM self-routing blocks, fitted on MMLU-Pro

Two small blocks from [**The Value of Thinking: Learning When to Reason in Hybrid
Decision Models**](https://arxiv.org/abs/2611.16961). Each one decides, before either mode runs, whether a question
is worth thinking about.

A block reads the hidden state at the readout position of the fast pass together
with eight statistics of the fast answer, and returns a distribution over the four
joint outcomes of the two modes. The fast mode's confidence and the value of
thinking are the two marginals of that distribution. The decision costs no forward
pass of its own, because it reads a state the fast pass has already produced.

Paper: <https://arxiv.org/abs/2611.16961>
Code: <https://github.com/zhengxuyu/HDM-code>

| | `head-4b-mmlu-pro` | `head-27b-mmlu-pro` |
|---|---|---|
| backbone | `Qwen/Qwen3.5-4B` | `Qwen/Qwen3.8-27B` |
| readout | `coded` | `coded` |
| layer | `-1`, forced | `-1`, forced |
| projection | none, raw hidden state | none, raw hidden state |
| input width | 2560 + 8 | 5120 + 8 |
| size | 2.6 MB | 5.3 MB |

## What they were fitted on

MMLU-Pro, 1,154 questions over eleven subject categories, with chemistry, economics
and psychology held out: 346 questions the block never saw, split at the level of the
category rather than the question. Supervised objective, seed 0. Fitted once and used
unchanged on every other benchmark in the paper.

| on the held-out categories | 4B | 27B |
|---|---|---|
| fast mode accuracy | 0.543 | 0.650 |
| slow mode accuracy | 0.853 | 0.887 |
| thinking helps | 0.335 | 0.272 |
| thinking hurts | 0.026 | 0.035 |
| AUROC, value of thinking against where thinking helps | 0.804 | 0.879 |
| AUROC, fast answer correct | 0.797 | 0.868 |

Read the last two rows beside the baselines in the report files here. On the 4B a
probe over the eight statistics alone scores 0.815 on the first, slightly above the
block, and the fast mode's own top probability scores 0.841 on the second. The raw
last layer is read anyway, for a reason that is not accuracy: every transformer has
a last layer, and a projection fitted on one backbone's features does not transfer
to another.

## Using one

`hde` is the engine from the code repository above; `pip install -e .` there puts
it on the path.

```python
from huggingface_hub import hf_hub_download
from hde.core.block import DecisionHead

path = hf_hub_download("fredreick/HDM-blocks", "head-4b-mmlu-pro.safetensors")
block = DecisionHead.load(path)
```

Serving refuses a pairing the block was not fitted for: the model id, the revision,
the readout and the hidden size are all compared before the block is attached, so a
block served against the wrong backbone raises rather than reading noise.

## A block is fitted to its features, not to a task

Changing the backbone, the layer, the readout or the numeric precision changes the
distribution these were fitted on, and the calibration goes with it. A block is a few
hundred kilobytes and fits in minutes, so refitting is the ordinary move.

## Citation

```bibtex
@article{yu2026valueofthinking,
  title         = {The Value of Thinking: Learning When to Reason in Hybrid Decision Models},
  author        = {Yu, Zhengxu and Xu, Ke and Fang, Meng and Ammar, Haitham and
                   Luo, Weilin and Wang, Jun},
  journal       = {arXiv preprint arXiv:2611.16961},
  year          = {2026},
  eprint        = {2611.16961},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI}
}
```