Q4: add the MTP drafting-vocabulary ranking (mtp.draft_rank), weights unchanged
Adds the MTP drafting-vocabulary ranking to Qwen3.6-35B-A3B-Q4.base. No weight changes.
What changed
- One tensor added to the MTP head section:
mtp.draft_rank, 248,320 token ids as f32, most frequent first, 993 KB. It's the rankingtools/mtp_draft_ranking.pybuilt for Qwen3.6-35B-A3B. Its sources are chat, reasoning, code and web text plus the model's own greedy outputs; SPEED-Bench is not a source. draft_vocab_size: 65536added to the speculator config.- It was grafted onto the published file rather than reconverted. The ranking is appended past the end of the weights and spliced into the header. All 20,448,493,568 weight bytes and every other header byte are identical to the current file (sha256
7171b37f…881b). Same tensor name, layout and checksum (xxh64) asbase-convert --mtp-draft-rankingwrites.
Why
With --speculate mtp-head, BaseRT's MTP head scored every drafted token against all 248,320 rows of the output matrix (0.32 GB per draft). With the ranking it scores the top 65,536 plus every special token (75.5 MB). Verify is unchanged, so the committed output is too.
Measured on an M5 Pro
One request at a time, 13 SPEED-Bench-style prompts, 256 tokens, greedy, decode tok/s:
| this file | current file | |
|---|---|---|
| MTP, BaseRT 0.3.0 library | 150.6 | 131–134 |
| MTP, with baseRT-internal PR #748 | 169.0 | 132.5 |
| No speculation | unchanged (logits bitwise identical) |
Acceptance is about 75% either way. With --spec-bitexact, MTP output is identical to plain decode on 13/13 prompts.
⚠️ Merge together with the catalog
The baseRT hub catalog pins this file's size and sha256 (base-convert/crates/base-hub/catalog.json, in the public basecompute/baseRT repo and bundled in each release). basert pull refuses a mismatch. Update that entry when this merges:
- size:
20515667968→20516661248 - sha256:
7171b37ffccdf16dfd920f55be766e6faffa31f64e04ca9cd0064943bf72881b→c57db6a481ed630859040cf65405bb591c8de8a4dc3e62c3f70979e91669b09b