Kimi K3.1 Pro DSpark
DSpark draft model for speculative decoding with Kimi K3.1 Pro. This is our latest checkpoint that replaces Kimi-K3.1-Pro-DSpark-preview.
This checkpoint is a joint effort with the Red Hat AI speculators team, using the vLLM project's speculators library.
| Component | Value |
|---|---|
| Layers | 5 dense decoder layers, 2.5B parameters (BF16) |
| Attention | Grouped-query attention with a 2,048-token sliding window in every layer; 96 query heads, 16 KV heads, head dim 64 |
| Hidden size / FFN | 7168 / 14336 |
| Input | hidden states from 5 target layers |
| Draft block | 7 tokens per forward pass |
| Heads | Markov head (rank 512), confidence head; embedding and LM head shared with the target |
| Context | 1,048,576 tokens; each draft layer attends to the last 2,048 |
Sliding-window attention
Every draft layer uses a 2,048-token sliding window, so the draft's attention cost and KV cache stay fixed as the context grows. This keeps drafting efficient at long context, up to Kimi K3.1 Pro's 1M tokens.
Evaluation settings
- Server:
--speculative-config '{"method":"dspark","model":"Inferact/Kimi-K3.1-Pro-DSpark","num_speculative_tokens":7,"draft_sample_method":"probabilistic","rejection_sample_method":"block"}' - Client: chat completions, temperature 1.0, top_p 0.95, full test set per benchmark.
- Metric: acceptance length = 1 + accepted draft tokens / drafts (counts the bonus token).
Results
| Benchmark | Prompts | Max tokens | Acceptance length | Accept rate, positions 1-7 |
|---|---|---|---|---|
| GSM8K | 1,319 | 1,024 | 5.566 | 0.91 0.82 0.72 0.64 0.56 0.49 0.43 |
| HumanEval | 164 | 1,024 | 4.660 | 0.86 0.71 0.59 0.49 0.40 0.34 0.28 |
| MBPP | 974 | 1,024 | 4.376 | 0.84 0.68 0.55 0.44 0.35 0.28 0.23 |
| MATH-500 | 500 | 2,048 | 4.445 | 0.84 0.69 0.56 0.45 0.37 0.30 0.24 |
| SPEED-Bench low entropy (10K-token inputs) | 302 | 1,536 | 3.619 | 0.78 0.58 0.42 0.31 0.23 0.17 0.13 |
| SPEED-Bench coding | 80 | 4,096 | 4.159 | 0.81 0.64 0.50 0.40 0.32 0.27 0.22 |
| MT-Bench | 80 | 1,024 | 3.393 | 0.76 0.54 0.38 0.27 0.20 0.14 0.11 |
| SWE-bench Pro | 642 | 2,048 | 3.302 | 0.75 0.53 0.37 0.26 0.18 0.13 0.09 |
| AIME 2026 | 30 | 4,096 | 3.429 | 0.77 0.56 0.40 0.28 0.20 0.14 0.10 |
| Pooled | 3.864 | 0.79 0.61 0.46 0.35 0.27 0.21 0.17 |
Long context
| Benchmark | Prompts | Prompt tokens | Max tokens | Acceptance length | Accept rate, positions 1-7 |
|---|---|---|---|---|---|
| AA-LCR | 100 | 95K mean | 1,024 | 3.935 | 0.81 0.64 0.49 0.37 0.27 0.21 0.15 |
| BEAM 128K | 400 | 97K-125K | 2,048 | 3.516 | 0.78 0.58 0.41 0.29 0.20 0.14 0.10 |
| BEAM 1M | 700 | 826K-1.04M | 2,048 | 3.421 | 0.77 0.56 0.40 0.28 0.19 0.13 0.09 |
- Downloads last month
- 123
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support