m8than/GLM-5.2-DFlash2-B12

GLM-5.2 DFlash2 B12 Furnace

An experimental 2.29B-parameter BF16 DFlash2 draft model for speculative decoding with GLM-5.2-FP8 in Furnace. This is the final, two-epoch checkpoint, selected at optimizer step 400. It is not a standalone language model, and the target model is not included.

The 12-token block contains 11 draft positions and one target token. A block size of 12 does not mean 12 accepted tokens or a 12x speedup. The measured live draft acceptance was 17.08%, with 2.87 output tokens per verification. This did not demonstrate an improvement over the preceding local checkpoint.

Architecture and compatibility

• Architecture: DFlash2DraftModel, including grouped dynamic convolutions and a learned candidate selector; not the original DFlashDraftModel architecture.
• Six layers, hidden size 6,144, intermediate size 12,288, 64 query heads, eight KV heads, and head dimension 96.
• Target feature layers: [1, 16, 31, 45, 60, 75].
• Convolution kernel size 2, group size 16; selector rank 256 and top-k 16.
• 2,289,554,560 parameters, stored in 96 BF16 tensors.

Use a Furnace runtime that supports this exact DFlash2 architecture and GLM-5.2 feature layout. Loading it as a generic Qwen3 model or assuming compatibility with an arbitrary upstream SGLang release is not supported. The qwen3 config family describes the draft backbone, not the required target model.

Example additions to an otherwise configured GLM-5.2 launch:

--model-path zai-org/GLM-5.2-FP8 \
--speculative-algorithm DFLASH \
--speculative-draft-model-path m8than/GLM-5.2-DFlash2-B12 \
--speculative-dflash-block-size 12 \
--speculative-draft-attention-backend triton \
--speculative-draft-kv-cache-dtype bfloat16 \
--attention-backend dsa \
--dsa-prefill-backend aiter \
--dsa-decode-backend aiter

The measured serving setup used eight AMD MI325X GPUs, target TP8, BF16 KV cache, an 8,192-token context limit, and up to four concurrent requests. Use the target model's tokenizer. Target weights, embeddings, and output head are provided by the target runtime, not this draft checkpoint.

Training and provenance

The original backbone donor was SubconsciousDev/glm-5.2-fp8-dflash-v2 at revision b45e4c46ed2902771d47c2cd09ee17870bb8be24. Despite its repository name, that donor used the original DFlash architecture. The local conversion retained 69 backbone tensors and initialized the new DFlash2 convolution and selector tensors. It then underwent a local 512-step pilot and a 5,000-step training run; the checkpoint selected from the latter was step 4,000.

The final adaptation released here warm-started from that selected checkpoint:

• 1,600 varied examples, two epochs, 400 new optimizer steps, global batch 8.
• 2,997,462 sequence tokens, including 2,002,397 supervised assistant tokens in the unique corpus; 3,200 example presentations over both epochs.
• Conversation, writing, multi-turn, supplied-context tasks, code, math, science, logic, and eight non-English languages; 457 multi-turn examples and 316 thinking examples.
• Every final-adaptation answer was decoded locally by Furnace GLM-5.2-FP8. Source-dataset assistant answers were discarded for this adaptation. System/user/template tokens and prior turns were excluded from loss.
• Frozen teacher revision: f33c6dc501ee5a2c7e35155653b1b1abbc320951.
• Maximum training context 8,192; 128 anchors per example; learning rate 3e-5; 4% warmup; checkpoints and validation at steps 200 and 400.
• Hard-label and hidden-state distillation, not soft-logprob KL distillation.

Earlier local stages had different data mixtures; the final adaptation's teacher-only label guarantee should not be attributed to the entire lineage. Prompt-source identifiers, revisions, and reported licenses are recorded in training_metadata.json. No training examples or private conversation are included in this repository.

Evaluation

The same 48 short serving probes were used for the predecessor and this release, covering multiple topics, languages, multi-turn prompts, thinking, and sampling. Each request had a 256-token generation cap. Counts are pooled across requests; these are a small diagnostic suite, not a comprehensive benchmark.

Checkpoint	Draft acceptance	Output tokens per verification
Previous local checkpoint	17.1485%	2.8739
This release, epoch 2 / step 400	17.0767%	2.8670

For this release, 8,051 of 47,146 proposed draft tokens were accepted over 4,286 verification iterations, producing 12,288 output tokens. Acceptance is accepted draft tokens divided by proposed draft tokens; it is not the probability of accepting an entire block. Output tokens per verification is not a measured wall-clock speedup. See metrics.json for aggregate counts and the draft-depth histogram reconstructed and checked against saved responses.

The held-out offline prefix score was 4.2123 at epoch one and 4.2256 at epoch two. Those offline scores are not live verifier acceptance. No live epoch-one result is claimed, and the private conversation-specific evaluation is not part of this release.

License and release limitations

The target model declares an MIT license, but the pinned donor draft and its immediate predecessor have no declared license or license file. This release does not assign a new license to inherited donor weights or infer permission from the target model's license. The donor's redistribution terms remain unresolved. Review the donor terms and the mixed prompt-source licenses before redistribution or broader use.

The model remains experimental: acceptance is modest, long-context capability is not established by the config's maximum-position value, and no production speedup, 12-token acceptance, or GLM-5.3 compatibility is claimed.