Qwen3.6-35B-A3B-DFlash2 for baseRT

The incoai DFlash 2 block drafter for Qwen3.6-35B-A3B (block 8, 8 taps, full-vocabulary drafts), packed as a baseRT .base sidecar. It drafts for basecompute/Qwen3.6-35B-A3B.

File Scheme Size Notes
Qwen3.6-35B-A3B-DFlash2-F16.base sidecar kept f16 throughout 1,052,921,856 B basert convert <snapshot> --target base-q4 (no --quantize-drafter)
Qwen3.6-35B-A3B-DFlash2-Q4.base base-q4 drafter layers and tap projections, f16 heads and norms 480,137,216 B basert convert <snapshot> --target base-q4 --quantize-drafter; the variant measured below

Source: incoai/Qwen3.6-35B-A3B-DFlash2 revision 51ef7b6923ad6c14cb1bb41c37a9041446496ab6, Apache-2.0 (LICENSE carried over). Converted with base-convert 0.4.4.

Use

basert pull basecompute/Qwen3.6-35B-A3B
basert pull basecompute/Qwen3.6-35B-A3B-DFlash2
basert serve <Qwen3.6-35B-A3B model.base> --speculate dflash:basecompute/Qwen3.6-35B-A3B-DFlash2

--speculate auto takes it once it is in the hub cache. The default policy is the measured one; set BASERTD_SPEC_PROMPT_REP_FLOOR=0 for RAG and summarization workloads (the prompt-time repetition gate otherwise decodes long, low-repetition prompts plainly).

Measured (baseRT 0.4.4, M6 mini 32 GB, SPEED-Bench 2 prompts x 11 categories, greedy, 256 output tokens)

Lanes Plain ms/token DFlash2, shipped policy Best fixed depth
1 13.0 1.52x K=7, 1.54x
2 18.3 1.32x K=3 or K=7, 1.33x
4 29.2 1.26x (149 aggregate tok/s vs 121) K=3, 1.28x
1, 6,000-token prompt 13.3 1.28x with the prompt gate off K=7

Acceptance 0.53–0.54 per draft, 0.71 accepted drafts per output token. Every draft is verified against the target, so the output is the target's on greedy decoding up to the engine's multi-row verify rounding (documented in baseRT's OPERATIONS.md, "Speculative decoding policy").

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for basecompute/Qwen3.6-35B-A3B-DFlash2

Finetuned
(1)
this model