JunHill/gui-planner-dflash2

GUI Planner DFlash2 Draft

This is a DFlash2 draft model, fully finetuned for a frozen Qwen3.8-27B GUI planner. It is used alongside the target in a speculative decoding engine; it is not a standalone planner or language model. The target model, tokenizer, image processor and chat template are not included.

The released checkpoint completed two epochs and 126 optimizer updates, updating all 1,924,404,480 draft parameters without LoRA. The target, shared embedding and language-model head stayed frozen. Training used block size 8. The same weights achieved their highest observed batch-one HTTP output throughput at inference block size 12: 68.51 tokens/s, or 1.550× the measured vanilla control.

These measurements do not establish lossless equivalence: generated sequences and parsed tool calls sometimes differ from vanilla inference. See the agreement metrics and limitations below.

Origin and compatibility

Initialization: z-lab/Qwen3.8-27B-DFlash2, revision 50307d4c4cde6860d4eee73e2547cd786fe8e8a4. This is full adaptation of the original draft, not a merge of a previous LoRA adapter.

The architecture retains five layers, hidden size 5,120, intermediate size 17,408, a 2,048-token sliding window, target feature taps [5, 19, 33, 47, 61], two-tap dynamic convolutions with group size 16, and a top-16 candidate selector with rank 256. The mask token ID is 248070.

Use the matching frozen Qwen3.8-27B planner target and its tokenizer, processor and chat template. The benchmark used the same original screenshots, tools and conversation history for all inference methods. Compatibility and speed with other target finetunes, prompt formats or preprocessing settings have not been measured.

Training

The source collection contained 2,500 GUI planning examples. Connected groups of related cases/runs, identical normalized tasks within an app, and identical complete multimodal inputs were assigned together, producing 1,999 train / 251 validation / 250 test examples across 205 connected groups. Apps may overlap between splits; this is not an unseen-app evaluation. The underlying examples are not distributed with this model.

Only train and validation inputs received greedy teacher responses. Training uses the frozen target's raw generated token IDs. Target features were computed from each full prompt and response; the feature cache retained the last 2,048 prompt tokens and the complete response, preserving absolute position offsets. No test input was used for training or validation. Inference block-size selection later used the test measurements and is exploratory.

Setting	Value
Hardware	8 NVIDIA H800 GPUs, 8 DDP ranks
Trainable scope	All draft backbone, dynamic-convolution and candidate-selector parameters
Epochs / optimizer steps	2 / 126
Training block size	8, including the known anchor token
Global batch size	32; gradient accumulation 4 per rank
Response anchors	8 random valid anchors per example
Optimizer	AdamW, learning rate 2e-5, betas (0.9, 0.95), weight decay 0.1
Schedule	4% warmup, then cosine decay
Gradient clipping / seed	1.0 / 42
Objective	Response-only full-vocabulary CE weighted by exp(-j/2), plus 0.1 × strict top-16 selector CE
Precision	FP32 draft/master weights and optimizer; BF16 autocast and frozen shared weights; BF16 serving export

These are pretrained-draft adaptation settings, not a reproduction of training from scratch in the DFlash paper. Validation CE decreased from 1.314474 before adaptation to 0.833334 after epoch 1 and 0.812287 after epoch 2.

Offline evaluation

Evaluation used 250 test inputs, three timed repeats after two warmups, and batch size 1 per TP2 replica. Four independent two-GPU replicas used eight H800 GPUs. Throughput is total output tokens divided by summed request-generation time per replica; it is not aggregate eight-GPU throughput.

All methods used greedy generation, seed 42, thinking disabled, at most 8,192 output tokens and context length 65,536. The original vanilla control is shared across block-size comparisons.

Method	Inference block	Acceptance length	Output tokens/s	Speedup vs vanilla	Exact sequence agreement	Parsed tool-call agreement
Vanilla target	—	—	44.19	1.000×	100.0%	100.0%
Original pretrained draft	8	3.840	66.73	1.510×	60.4%	84.0%
This checkpoint	8	4.125	67.45	1.526×	60.4%	84.0%
This checkpoint	12	4.311	68.51	1.550×	55.2%	81.2%
This checkpoint	16	4.232	67.33	1.524×	60.4%	85.2%

Acceptance length is 1 + accepted draft tokens / verification rounds, including the bonus token. Blocks 8, 12 and 16 request 7, 11 and 15 speculative tokens, respectively. All configurations completed 750 timed requests with zero errors and zero truncated responses.

The HTTP timer includes upload, image preprocessing, prefill, decode and response parsing. It excludes file/base64 preparation, server initialization, warmup, metrics polling and report writing. Server first-to-last-token intervals averaged 466.07 / 469.51 / 501.19 ms per request for blocks 8 / 12 / 16. Thus block 12's small overall throughput gain does not demonstrate faster decoding. Output lengths also differ between configurations.

Sequence and tool-call agreement use the first repeat. Tool-call comparison ignores generated call IDs and canonicalizes JSON arguments; empty tool-call lists may match even when text differs. No GUI environment was executed, so agreement is not task-success rate. The reason for cross-method output differences has not been isolated. Runs were measured separately, with no statistical-significance claim.

Serving

The evaluated stack was Python 3.12, PyTorch 2.13.0+cu129, Transformers 5.19.0 and vLLM 0.32.0+cu129. It requires a vLLM build with DFlash2 candidate-selector support and VLLM_USE_V2_MODEL_RUNNER=1. The tested V1 runner did not support this draft architecture.

The following command shows the core serving settings for the highest observed throughput. Set the two variables to the matching target and this draft's local directory or Hub repository ID:

export TARGET_MODEL="/path/to/matching/planner-target"
export DRAFT_MODEL="JunHill/gui-planner-dflash2"
export VLLM_USE_V2_MODEL_RUNNER=1

vllm serve "${TARGET_MODEL}" \
  --served-model-name target \
  --dtype bfloat16 \
  --tensor-parallel-size 2 \
  --max-model-len 65536 \
  --max-num-batched-tokens 4096 \
  --max-num-seqs 1 \
  --limit-mm-per-prompt '{"image":4,"video":0}' \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --generation-config vllm \
  --gpu-memory-utilization 0.90 \
  --no-enable-prefix-caching \
  --mm-processor-cache-gb 0 \
  --seed 42 \
  --speculative-config "{\"method\":\"dflash\",\"model\":\"${DRAFT_MODEL}\",\"num_speculative_tokens\":11,\"draft_sample_method\":\"greedy\"}"

Use request-level temperature=0, max_tokens=8192 and chat_template_kwargs={"enable_thinking": false} to match the benchmark's generation settings.

The original benchmark also used a shared server wrapper that capped whitespace at four characters for strict Qwen XML tool-call schemas representing empty objects (properties={}, additionalProperties=false). That same policy was applied to vanilla and every draft configuration. The wrapper is included as serve_planner.py: after downloading the repository, replace vllm serve "${TARGET_MODEL}" in the command above with python serve_planner.py --model "${TARGET_MODEL}" to apply it. The remaining flags stay the same. Reproducing the published measurements also requires the matching request formatting, hardware and model versions.

For the trained block size, set num_speculative_tokens to 7. Blocks 12 and 16 are inference extrapolation from training block 8. The released checkpoint configuration remains unchanged; select a larger block through the serving configuration. Because the draft uses bidirectional attention within the query block, increasing its length can change earlier draft predictions too.

References and license

• Original DFlash2 model and usage
• DFlash implementation
• DFlash paper

The upstream draft is distributed under Apache 2.0. This derivative draft is released under Apache 2.0. Target weights and the training/evaluation examples are separate assets and are not included in this repository.