RedHatAI/GLM-5.2-speculator.dspark

This is a preliminary (and subject to change) DSpark speculator model for nvidia/GLM-5.2-NVFP4.

It was trained using the Speculators library on the mgoin/open-perfectblend-glm5.2-regen dataset.

Note:

It was validated on Nvidia B200, other hardware validation pending.

We are continuing to train this model and will update with more evaluations and new weights in the future.

Training Details

Launch training

scripts/train.py \
  --verifier-name-or-path RedHatAI/GLM-5.2-NVFP4-FP8 \
  --data-path ./output/dspark_glm52 \
  --vllm-endpoint http://localhost:8000/v1 \
  --save-path ./output/dspark_glm52/checkpoints \
  --epochs 1 \
  --lr 3e-4 \
  --total-seq-len 8192 \
  --speculator-type dspark \
  --block-size 8 \
  --max-anchors 1024 \
  --num-layers 3 \
  --target-layer-ids 2 20 39 58 75 \
  --markov-rank 256 \
  --markov-head-type vanilla \
  --enable-confidence-head \
  --confidence-head-with-markov \
  --loss-fn '{"ce": 0.1, "tv": 0.9}' \
  --confidence-head-alpha 1.0 \
  --fsdp-shard \
  --logger tensorboard \
  --log-dir ./output/dspark_glm52/logs \
  --on-missing generate \
  --on-generate delete \
  --checkpoint-freq 0.01 \
  --request-timeout 60 \
  --max-retries 2

Deployment

Deploy with vLLM using the speculator as a draft model.

vllm serve nvidia/GLM-5.2-NVFP4 \
  --max-model-len 20480 \
  --spec-model RedHatAI/GLM-5.2-speculator.dspark \
  --spec-method dspark \
  --spec-tokens 7 \
  -tp 8

Preliminary Evaluations

Per-Position Acceptance Rate

Dataset	Pos 0	Pos 1	Pos 2	Pos 3	Pos 4	Pos 5	Pos 6	Avg. Length
HumanEval	77.8%	59.1%	44.6%	33.9%	26.1%	20.3%	15.8%	3.78
math_reasoning	87.5%	76.6%	67.6%	59.2%	51.8%	45.1%	39.5%	5.27
qa	70.7%	48.5%	33.7%	23.7%	17.1%	11.9%	8.4%	3.14
question	73.7%	52.6%	38.1%	28.5%	21.6%	16.7%	13.2%	3.44
rag	78.5%	61.3%	48.7%	38.9%	30.7%	24.2%	19.2%	4.01
summarization	77.9%	58.0%	43.0%	31.7%	22.8%	16.2%	11.4%	3.61
tool_call	71.9%	51.1%	37.1%	27.6%	21.4%	16.8%	13.2%	3.39
translation	76.8%	57.4%	43.5%	33.6%	26.1%	20.0%	15.1%	3.72
writing	74.0%	52.2%	38.0%	28.1%	21.3%	16.7%	13.1%	3.44