Whittle 35B-A3B DFlash2

A focused draft for Whittle.

526M parameters · F16 GGUF · v1.0.0 · Apache-2.0

Download weights · Training data · Release notes

DFlash2 draft for Whittle Q5_K_M. A DFlash2 drafting adapter for speculative decoding with Whittle-Qwen-3.8-35B-A3B.

Run it alongside the Whittle target model in a compatible DFlash2 runtime. Training updates the draft parameters while the Whittle target stays frozen. The 35B target is downloaded separately.

At a glance

	Specification
Draft	526,251,520 parameters · 6 layers · 16-token training block
Draft initialization	Ornith 1.5 DFlash2
Evaluated target	Whittle Q5_K_M GGUF
Download	F16 GGUF · 1,063,709,024 bytes · 0.991 GiB
Languages	Russian · English
License	Apache-2.0

Repository branches

Branch	Contents
`main`	The current model card and repository guide.
`v1.0.0`	Weights, release changelog, training code, usage guide and evaluation reports.

Training data

The DFlash2 drafter was trained using DFlash TraceMix Lite and 1,152 first-user-turn tasks selected from Tulu 3 SFT Mixture: 576 Russian and 576 English, balanced across code, math and reasoning, text writing, and knowledge.

Tulu reference answers were not used as training labels. Whittle Q5 generated the responses and hidden features used to train the DFlash2 drafter.

Training stage	Data	Train	Validation
Project adaptation	Project Code	512	64
Mixed continuation	OpenMix	1,276 usable	128

The Tulu selection comprised 1,024 training and 128 validation tasks. Four training traces were too short, leaving 1,020 usable Tulu training traces. Adding 256 replayed Project Code training records produced 1,276 training and 128 validation examples for OpenMix. The training guide and data sources describe the recipe.

Recorded local performance

Configuration	Decode tokens/s	End-to-end tokens/s	Draft acceptance
Whittle Q5, no draft	53.60	50.13	—
Whittle DFlash2 v1.0.0	74.08	67.62	59.53%

Measured on 24 long-generation requests: 12 tasks, two passes, up to 1,024 output tokens. Linux; V100 32 GB for target computation and draft, RTX 3060 12 GB for target n-gram memory. Custom llama.cpp runtime; maximum three proposed tokens and BF16 draft KV cache.

Outputs differed from the no-draft control, so this comparison does not establish equivalent quality. Validation pools were reused during checkpoint selection. Full results: EVALUATION.json.

Runtime and limitations

Evaluated with a modified llama.cpp fork. Other target quantizations, stock llama.cpp, SGLang, vLLM and Apple Metal were not validated for this artifact.
Exact greedy equivalence to no-draft generation remains unconfirmed.
Both 64K marker checks failed with this draft. A filled 256K context was not tested.
All 24 long-generation answers reached the output limit; generated code was not executed.

See USAGE.md for runtime parameters and PROVENANCE.json for lineage.

_{Adaptation by Egor Prokhorov · Published by biscet · License & notices}