HDM-Qwen3.5-4B

The self-routing block of HDM for the frozen backbone Qwen/Qwen3.5-4B, from The Value of Thinking: Learning When to Reason in Hybrid Decision Models.

This repository holds only the block (658,692 parameters, 2.7 MB). The backbone is not included and is not modified.

Paper: https://arxiv.org/abs/2611.16961 Code: https://github.com/zhengxuyu/HDM-code Other size: fredreick/HDM-Qwen3.8-27B

What it does

The backbone first answers a question in one forward pass, with no generated tokens. The block reads the hidden state of that pass, together with eight statistics of the fast answer, and predicts which of four things is true:

  • the fast answer is right and thinking is right,
  • the fast answer is right and thinking is wrong,
  • the fast answer is wrong and thinking is right,
  • both are wrong.

The value of thinking is P(fast wrong, thinking right) minus P(fast right, thinking wrong). A question is sent to thinking when that value is above a price λ. λ is set per request and is not used in fitting.

Requirements

backbone Qwen/Qwen3.5-4B
serving backend SGLang 0.5.9 for the results below. The transformers backend serves the same block, but its features are not identical, so a question can be routed differently there
readout coded
layer -1, raw hidden state, width 2560 + 8

Serving compares the model id, the revision, the readout and the hidden size before the block is attached, and raises if they do not match.

Using it

git clone https://github.com/zhengxuyu/HDM-code
cd HDM-code
uv sync

huggingface-cli download fredreick/HDM-Qwen3.5-4B head-4b-mmlu-pro.safetensors --local-dir weights/

hde-serve --model Qwen/Qwen3.5-4B \
          --decision-head weights/head-4b-mmlu-pro.safetensors \
          --backend sglang --readout coded --preload --port 8000 \
          --lambda 0 --think-budget 4096

--lambda and --think-budget set the default price and thinking budget; a request can override them in its routing field.

To load the block on its own:

from huggingface_hub import hf_hub_download
from hde.core.block import DecisionHead

path = hf_hub_download("fredreick/HDM-Qwen3.5-4B", "head-4b-mmlu-pro.safetensors")
block = DecisionHead.load(path)

What it was fitted on

MMLU-Pro, 1,154 questions over eleven subject categories. Chemistry, economics and psychology were held out: 346 questions the block never saw. Supervised objective, seed 0, thinking budget 4,096 tokens. Fitted once and used unchanged on every other benchmark in the paper. The fitting report is in head-4b-mmlu-pro-report.json.

Results

"Random" is the accuracy expected if the same number of questions were sent to thinking at random.

MMLU-Pro, the 346 held-out questions. Never thinking: 54.3%. Always thinking: 85.3%.

price λ sent to thinking accuracy random thinking tokens, as a share of always thinking
0.1 52.9% 81.5% 70.7% 69%
0.05 58.4% 83.5% 72.4% 75%
0 66.2% 84.4% 74.8% 81%
−0.05 93.9% 85.0% 83.4% 94%
−0.1 96.8% 84.4% 84.3% 97%

JevBench, 231 public items. The block was not fitted on JevBench. Never thinking: 76.2%. Always thinking: 93.1%.

price λ sent to thinking accuracy random thinking tokens, as a share of always thinking
0.1 16.5% 83.5% 79.0% 33%
0.05 17.7% 83.5% 79.2% 35%
0 30.3% 85.7% 81.3% 47%

Limitations

  • Fitted on multiple-choice questions only. Other question types are untested.
  • On JevBench every price below zero sends all questions to thinking, so only prices of zero and above give a choice there.
  • The block also sends most of the questions that thinking cannot fix to thinking, and those tokens are wasted.
  • Each table is one run.
  • Changing the backbone, the layer, the readout or the numeric precision changes the features the block was fitted on. Refit it in that case; fitting takes minutes.

Citation

@article{yu2026valueofthinking,
  title         = {The Value of Thinking: Learning When to Reason in Hybrid Decision Models},
  author        = {Yu, Zhengxu and Xu, Ke and Fang, Meng and Ammar, Haitham and
                   Luo, Weilin and Wang, Jun},
  journal       = {arXiv preprint arXiv:2611.16961},
  year          = {2026},
  eprint        = {2611.16961},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for fredreick/HDM-Qwen3.5-4B

Finetuned
Qwen/Qwen3.5-4B
Finetuned
(1,540)
this model

Paper for fredreick/HDM-Qwen3.5-4B