license: apache-2.0
library_name: safetensors
tags:
- adaptive-inference
- routing
- decision-model
base_model:
- Qwen/Qwen3.5-4B
- Qwen/Qwen3.8-27B
HDM self-routing blocks, fitted on MMLU-Pro
Two small blocks from The Value of Thinking: Learning When to Reason in Hybrid Decision Models. Each one decides, before either mode runs, whether a question is worth thinking about.
A block reads the hidden state at the readout position of the fast pass together with eight statistics of the fast answer, and returns a distribution over the four joint outcomes of the two modes. The fast mode's confidence and the value of thinking are the two marginals of that distribution. The decision costs no forward pass of its own, because it reads a state the fast pass has already produced.
Paper: https://arxiv.org/abs/2611.16961 Code: https://github.com/zhengxuyu/HDM-code
head-4b-mmlu-pro |
head-27b-mmlu-pro |
|
|---|---|---|
| backbone | Qwen/Qwen3.5-4B |
Qwen/Qwen3.8-27B |
| readout | coded |
coded |
| layer | -1, forced |
-1, forced |
| projection | none, raw hidden state | none, raw hidden state |
| input width | 2560 + 8 | 5120 + 8 |
| size | 2.6 MB | 5.3 MB |
What they were fitted on
MMLU-Pro, 1,154 questions over eleven subject categories, with chemistry, economics and psychology held out: 346 questions the block never saw, split at the level of the category rather than the question. Supervised objective, seed 0. Fitted once and used unchanged on every other benchmark in the paper.
| on the held-out categories | 4B | 27B |
|---|---|---|
| fast mode accuracy | 0.543 | 0.650 |
| slow mode accuracy | 0.853 | 0.887 |
| thinking helps | 0.335 | 0.272 |
| thinking hurts | 0.026 | 0.035 |
| AUROC, value of thinking against where thinking helps | 0.804 | 0.879 |
| AUROC, fast answer correct | 0.797 | 0.868 |
Read the last two rows beside the baselines in the report files here. On the 4B a probe over the eight statistics alone scores 0.815 on the first, slightly above the block, and the fast mode's own top probability scores 0.841 on the second. The raw last layer is read anyway, for a reason that is not accuracy: every transformer has a last layer, and a projection fitted on one backbone's features does not transfer to another.
Using one
hde is the engine from the code repository above; pip install -e . there puts
it on the path.
from huggingface_hub import hf_hub_download
from hde.core.block import DecisionHead
path = hf_hub_download("fredreick/HDM-blocks", "head-4b-mmlu-pro.safetensors")
block = DecisionHead.load(path)
Serving refuses a pairing the block was not fitted for: the model id, the revision, the readout and the hidden size are all compared before the block is attached, so a block served against the wrong backbone raises rather than reading noise.
A block is fitted to its features, not to a task
Changing the backbone, the layer, the readout or the numeric precision changes the distribution these were fitted on, and the calibration goes with it. A block is a few hundred kilobytes and fits in minutes, so refitting is the ordinary move.
Citation
@article{yu2026valueofthinking,
title = {The Value of Thinking: Learning When to Reason in Hybrid Decision Models},
author = {Yu, Zhengxu and Xu, Ke and Fang, Meng and Ammar, Haitham and
Luo, Weilin and Wang, Jun},
journal = {arXiv preprint arXiv:2611.16961},
year = {2026},
eprint = {2611.16961},
archivePrefix = {arXiv},
primaryClass = {cs.AI}
}