HDM-Qwen3.8-27B
The self-routing block of HDM for the frozen backbone Qwen/Qwen3.8-27B, from The Value of Thinking: Learning When to Reason in Hybrid Decision Models.
This repository holds only the block (1,314,052 parameters, 5.3 MB). The backbone is not included and is not modified.
Paper: https://arxiv.org/abs/2611.16961 Code: https://github.com/zhengxuyu/HDM-code Other size: fredreick/HDM-Qwen3.5-4B
What it does
The backbone first answers a question in one forward pass, with no generated tokens. The block reads the hidden state of that pass, together with eight statistics of the fast answer, and predicts which of four things is true:
- the fast answer is right and thinking is right,
- the fast answer is right and thinking is wrong,
- the fast answer is wrong and thinking is right,
- both are wrong.
The value of thinking is P(fast wrong, thinking right) minus P(fast right, thinking wrong). A question is sent to thinking when that value is above a price λ. λ is set per request and is not used in fitting.
Requirements
| backbone | Qwen/Qwen3.8-27B. Fitted against commit 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0; the backbone repository has been updated since |
| serving backend | SGLang 0.5.9 for the results below. The transformers backend serves the same block, but its features are not identical, so a question can be routed differently there |
| readout | coded |
| layer | -1, raw hidden state, width 5120 + 8 |
Serving compares the model id, the revision, the readout and the hidden size before the block is attached, and raises if they do not match.
Using it
git clone https://github.com/zhengxuyu/HDM-code
cd HDM-code
uv sync
huggingface-cli download fredreick/HDM-Qwen3.8-27B head-27b-mmlu-pro.safetensors --local-dir weights/
hde-serve --model Qwen/Qwen3.8-27B \
--decision-head weights/head-27b-mmlu-pro.safetensors \
--backend sglang --readout coded --preload --port 8000 \
--lambda 0 --think-budget 4096
--lambda and --think-budget set the default price and thinking budget; a request can override them in its routing field.
To load the block on its own:
from huggingface_hub import hf_hub_download
from hde.core.block import DecisionHead
path = hf_hub_download("fredreick/HDM-Qwen3.8-27B", "head-27b-mmlu-pro.safetensors")
block = DecisionHead.load(path)
What it was fitted on
MMLU-Pro, 1,154 questions over eleven subject categories. Chemistry, economics and psychology were held out: 346 questions the block never saw. Supervised objective, seed 0, thinking budget 4,096 tokens. Fitted once and used unchanged on every other benchmark in the paper. The fitting report is in head-27b-mmlu-pro-report.json.
Results
"Random" is the accuracy expected if the same number of questions were sent to thinking at random.
MMLU-Pro, the 346 held-out questions. Never thinking: 65.0%. Always thinking: 90.2%.
| price λ | sent to thinking | accuracy | random | thinking tokens, as a share of always thinking |
|---|---|---|---|---|
| 0.1 | 46.5% | 88.7% | 76.7% | 77% |
| 0.05 | 48.3% | 89.0% | 77.2% | 78% |
| 0 | 63.3% | 89.6% | 80.9% | 85% |
| −0.05 | 97.7% | 90.2% | 89.6% | 98% |
| −0.1 | 98.8% | 90.2% | 89.9% | 99% |
JevBench, 231 public items. The block was not fitted on JevBench. Never thinking: 87.9%. Always thinking: 99.1%.
| price λ | sent to thinking | accuracy | random | thinking tokens, as a share of always thinking |
|---|---|---|---|---|
| 0.1 | 10.8% | 92.6% | 89.1% | 25% |
| 0.05 | 13.4% | 93.5% | 89.4% | 28% |
| 0 | 53.7% | 98.7% | 93.9% | 82% |
Limitations
- Fitted on multiple-choice questions only. Other question types are untested.
- On JevBench every price below zero sends all questions to thinking, so only prices of zero and above give a choice there.
- The block also sends most of the questions that thinking cannot fix to thinking, and those tokens are wasted.
- Each table is one run.
- Changing the backbone, the layer, the readout or the numeric precision changes the features the block was fitted on. Refit it in that case; fitting takes minutes.
Citation
@article{yu2026valueofthinking,
title = {The Value of Thinking: Learning When to Reason in Hybrid Decision Models},
author = {Yu, Zhengxu and Xu, Ke and Fang, Meng and Ammar, Haitham and
Luo, Weilin and Wang, Jun},
journal = {arXiv preprint arXiv:2611.16961},
year = {2026},
eprint = {2611.16961},
archivePrefix = {arXiv},
primaryClass = {cs.AI}
}
Model tree for fredreick/HDM-Qwen3.8-27B
Base model
Qwen/Qwen3.8-27B