Q4: add the MTP drafting-vocabulary ranking (mtp.draft_rank), weights unchanged
Adds the MTP drafting-vocabulary ranking to Qwen3.8-27B-Q4.base. No weight changes.
What changed
- One tensor added to the MTP head section:
mtp.draft_rank, 248,320 token ids as f32, most frequent first, 993 KB. It's the Qwen3.8-27B ranking built withtools/mtp_draft_ranking.py; its tokenizer hashd868d01502157b74is this bundle's own. draft_vocab_size: 65536added to the speculator config.- It was grafted onto the published file with
tools/internal/graft_mtp_draft_rank.py(baseRT-internal #750), not reconverted. All 17,773,920,256 weight bytes and every other header byte are identical to the current file (sha25603978674…0526, matching the catalog pin).
Why
With --speculate mtp-head, the MTP head scored every drafted token against all 248,320 output rows, about 715 MB per draft. It now scores the top 65,536 plus every special token (189 MB). Verify is unchanged.
Measured on an M5 Pro
BaseRT 0.3.0 library through superfluid, one request at a time, 13 SPEED-Bench-style prompts, 256 tokens, greedy, decode tok/s (two runs each, in alternating order):
| this file | current file | |
|---|---|---|
| MTP | 32.8, 34.0 | 31.7, 31.4 |
| Acceptance | 71% | 69% |
| No speculation | unchanged (logits bitwise identical) | 18.6 |
The gain is smaller than on Qwen3.6-35B-A3B (+6% here vs +13%). On a dense 27B, each verify pass reads the whole 17.8 GB model, so drafting is a smaller share of the time.
With --spec-bitexact, MTP output equals plain decode on 12/13 prompts. The current file is also 12/13 (on a different prompt): bit-exact verify is guaranteed on routed-MoE models, not on this dense one. So the graft doesn't change that.
⚠️ Merge together with the catalog
The baseRT hub catalog pins this file's size and sha256 (base-convert/crates/base-hub/catalog.json, in the public basecompute/baseRT repo and bundled in each release). basert pull refuses a mismatch. Update that entry when this merges:
- size:
17841094656→17842087936 - sha256:
03978674a487c567f0e5378b725b8ee1226bf554e8ecafa72943a4e7b99a0526→527c522a441900029b133e76316fd1f58e1bcbd86c5341ccc5315ee4d9ed8de6