Kimi K3.1 Pro DSpark

DSpark draft model for speculative decoding with Kimi K3.1 Pro. This is our latest checkpoint that replaces Kimi-K3.1-Pro-DSpark-preview.

This checkpoint is a joint effort with the Red Hat AI speculators team, using the vLLM project's speculators library.

Component Value
Layers 5 dense decoder layers, 2.5B parameters (BF16)
Attention Grouped-query attention with a 2,048-token sliding window in every layer; 96 query heads, 16 KV heads, head dim 64
Hidden size / FFN 7168 / 14336
Input hidden states from 5 target layers
Draft block 7 tokens per forward pass
Heads Markov head (rank 512), confidence head; embedding and LM head shared with the target
Context 1,048,576 tokens; each draft layer attends to the last 2,048

Sliding-window attention

Every draft layer uses a 2,048-token sliding window, so the draft's attention cost and KV cache stay fixed as the context grows. This keeps drafting efficient at long context, up to Kimi K3.1 Pro's 1M tokens.

Evaluation settings

  • Server: --speculative-config '{"method":"dspark","model":"Inferact/Kimi-K3.1-Pro-DSpark","num_speculative_tokens":7,"draft_sample_method":"probabilistic","rejection_sample_method":"block"}'
  • Client: chat completions, temperature 1.0, top_p 0.95, full test set per benchmark.
  • Metric: acceptance length = 1 + accepted draft tokens / drafts (counts the bonus token).

Results

Benchmark Prompts Max tokens Acceptance length Accept rate, positions 1-7
GSM8K 1,319 1,024 5.566 0.91 0.82 0.72 0.64 0.56 0.49 0.43
HumanEval 164 1,024 4.660 0.86 0.71 0.59 0.49 0.40 0.34 0.28
MBPP 974 1,024 4.376 0.84 0.68 0.55 0.44 0.35 0.28 0.23
MATH-500 500 2,048 4.445 0.84 0.69 0.56 0.45 0.37 0.30 0.24
SPEED-Bench low entropy (10K-token inputs) 302 1,536 3.619 0.78 0.58 0.42 0.31 0.23 0.17 0.13
SPEED-Bench coding 80 4,096 4.159 0.81 0.64 0.50 0.40 0.32 0.27 0.22
MT-Bench 80 1,024 3.393 0.76 0.54 0.38 0.27 0.20 0.14 0.11
SWE-bench Pro 642 2,048 3.302 0.75 0.53 0.37 0.26 0.18 0.13 0.09
AIME 2026 30 4,096 3.429 0.77 0.56 0.40 0.28 0.20 0.14 0.10
Pooled 3.864 0.79 0.61 0.46 0.35 0.27 0.21 0.17

Long context

Benchmark Prompts Prompt tokens Max tokens Acceptance length Accept rate, positions 1-7
AA-LCR 100 95K mean 1,024 3.935 0.81 0.64 0.49 0.37 0.27 0.21 0.15
BEAM 128K 400 97K-125K 2,048 3.516 0.78 0.58 0.41 0.29 0.20 0.14 0.10
BEAM 1M 700 826K-1.04M 2,048 3.421 0.77 0.56 0.40 0.28 0.19 0.13 0.09
Downloads last month
123
Safetensors
Model size
2B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support