Download docs/performance.md from staccs/ds4-winhip: direct link, hf CLI and curl.
- Browser
- Download file 17.8 kB
-
https://huggingface.co/staccs/ds4-winhip/resolve/main/docs/performance.md
- Command line
-
hf download hf://staccs/ds4-winhip/docs/performance.md
-
curl -L -o performance.md https://huggingface.co/staccs/ds4-winhip/resolve/main/docs/performance.md
Performance references
The README table presents recorded whole-model prefill and decode examples. This guide explains their conditions and links to the original reports. Workloads, clocks, artifacts and sampling differ between rows; use the original protocol for a comparison.
Rates are tokens/second. K means 1,024 tokens; 8K + 64 means 8,192 input
tokens and 64 generated tokens. Prefill processes the prompt; decode generates
the continuation. A sweep with reused state measures incremental suffixes,
while a cold request computes the whole prompt. MTP changes the decode workload.
Throughput does not establish context, bank, cache or media qualification.
Qwen
Qwen3.8 Flash Next Base Q5 on one DGX Spark / GB10, with only the SSD-PLE format changed. The FP8 PLE guide owns the exact command, identities, paired results and numerical limits.
| PLE format | Prefill tok/s | Decode tok/s |
|---|---|---|
| BF16 | 1,245.1 | 28.59 |
| FP8 | 1,323.1 | 28.93 |
These cells are medians of three run means over 32 frontiers, from 2K to 64K. Each process uses one warm session, 2,048-token incremental prefills and 128 greedy outputs per frontier. The same frozen binary and aligned-Q8 owner serve both formats, with a 2 GiB PLE cache, 16 page workers and an 8,192-token chunk cap.
Curves show per-frontier medians; bands show observed min/max, not confidence intervals. MTP draft 2 was requested; one BF16 run autoquenched and remains in the data. FP8 changes embedding values and can change tokens and draft acceptance. This is the recorded automatic-policy result, not an isolated MTP speedup or a cold 64K/256K serving measurement.
Source: paired results and verification, raw CSVs and receipt.
Later cold-prefill and draft/prefix campaigns retain their own protocols: September 6, September 7, September 14.
Qwen Uncensored
The same paired card sweep records 1,310.8 prefill / 28.96 decode for Uncensored Q5 with FP8 PLE. It is a separate main model, measured with the same protocol. The README graph shows Base Q5. Swift's card reuses the Base Q5 graph as a reference; no Swift throughput is reported here.
Solar
Solar Open2 250B MXQ-v1, one GB10, cold ds4-bench requests, 64 greedy outputs,
4,096-token chunks, MTP off. The September 14 round 2
records 1,095.61 prefill / 17.43 decode at 8K and 943.18 / 13.01 at 64K.
Both are medians of three interleaved comparisons under the 300β2200 MHz
clock range; sampled SM clocks were 2190 MHz. Logits and IDs matched byte for byte.
The curve shows the separate round-1 FATTN_WS off/on comparison. The README uses the later round-2 result. These are cold benchmark requests, distinct from HTTP reuse and the earlier uncapped campaigns. Keep the recorded clock conditions when reproducing the result.
Motif
Motif-3 MQ87-88's August 21 C baseline
records 627.19 prefill / 15.06 decode at 8K + 64 on GB10. It uses a 4,096-token
chunk, aligned-Q8 VMM owner, greedy non-thinking generation and no speculation.
This is inherited v0.6.2-dfm evidence, not a fresh Rust-host measurement.
dots3
dots3-note MQ87's September 6 campaign records 604.3 prefill / 16.78 decode at 8K + 64, serial plain decoding on GB10. The same-binary kill-switch comparisons preserve the frontier argmax, top-10 and all 64 greedy IDs. The result does not qualify later opt-in banks or MTP.
K2
K2-Horizon MQ87's September 5 continuation scoreboard contains two all-default round-6 samples: 644.78 / 13.34 and 641.94 / 13.07 prefill/decode. The README retains their ranges rather than selecting the faster sample. These are fresh GB10 processes, 8K + 64, context 8,257, no MTP or reuse. The report defines the accepted arithmetic differences and clock conditions.
Inkling
Inkling Small MQ85GB's September 22 capped-clock result records 452.58 prefill / 13.07 decode for the retained router tile. Workload: cold 8K + 64, MTP off, chunk 2,048, three interleaved pairs on GB10. SM clocks were 2190β2197 MHz. The higher unlocked-clock figures are a separate result; contexts above 8K were not measured in this campaign.
Step
Step 3.7 Flash MQ83's capped-clock aggregate records 1,194.64 prefill / 22.77 decode in the matched final 2K + 64 comparison. It uses the Q8 MTP sidecar, draft 3, three fresh-process pairs and the same VMM owner under a 300β2200 MHz GB10 clock range. Logits and tokens match exactly. The separate 16K result changes the MTP trajectory and is not this table row.
Ling
Ling-3.0-flash-VL MQ-Q5's GB10 campaign records 1,889 prefill / 24.65 decode at 8K + 64 after PR #48, with busy SM clocks 2177β2197 MHz. The family has no MTP. Later PR #49/#50 results use different retained paths and are recorded separately.
This later card sweep uses one warm session per fresh process. Curves are per-frontier medians; bands are observed min/max over two #48 runs and three each for #49 and #50. The raw CSVs and receipt retain the workload and artifact identities.
MiMo
MiMo-V2.6-Flash-RL mixed quant's September 26 round 3 records 1,217.76 prefill / 24.44 decode, medians of three fresh samples per arm. Workload: 8K + 128, chunks 4,096, MTP/DFlash off, shared VMM owner, GB10 under a 300β2200 MHz clock range. All 152,576 logits and 128 tokens match exactly. The media campaign is separate.
MiMo MOPD
The September 30 MOPD round 2 records 1,206.290 prefill / 24.180 decode for the retained ON path at 8K + 128. Three rotated fresh-process rounds use chunk 4,096, a shared VMM owner and MTP/DFlash inactive. Busy clocks were 2184β2190 MHz within the 300β2200 range. The longer 8K + 1,024 check has equal OFF/ON decode medians; it does not reproduce the short-decode decrease. This is a separate artifact and workload from RL.
Naive
Naive-N0.5-Flash MQ87's October 1 paired index result
records 518.85 prefill / 18.85 ordinary decode for index2-on on GB10.
Three fresh processes per side use separate warmups, chunk 2,048 and 32
greedy outputs; all 915,456 checked logits and token/argmax decisions match
exactly. Sampled clocks are 2190β2197 MHz. Native source is ee28ed26;
the report pins each round's benchmark bytes and eager/actual-state proofs.
The September 30 index-packing pair
remains historical. These results do not qualify DSpark acceleration or
long-context serving; use the family guide for those limits.
GLM-5.3 Flash VQ
The 19-layer R13+8 release of GLM-5.3 Flash VQ
(model card, HF
71c73e7e: routed layers 3β17, 22, 27, 41 and 44 add an 8-bit residual
stage, Q4_K nonexperts unchanged) recorded 1,101.3 prefill / 30.93 decode
with two embedded MTP drafts in the
2Kβ64K sweep
(94d776ac). The 14-layer revision (layers 3β16, tag r13p8-l3-16)
measured 1,119.6 / 32.02 in the same interleaved runs and binary: prefill
β1.6% from five more layers' residual MMAs, a verify cycle +1.0% from 7.98
instead of 7.85 GB per token, and 2.171 instead of 2.226 tokens per verify
on these continuations. The 19-layer revision lowers held-out PPL from 4.042
to 3.895 and stops looping on long-context served answers (0 of 10 seeds
instead of 4); its 1M quote keeps one bank with 4,096-row prefill chunks.
The 14-layer revision (HF 35d3833d) recorded 1,113.7 / 31.81 in the
2Kβ64K sweep
(90517b6b). The previous R13 release measured 1,175.4 / 35.00 in the same
interleaved runs and binary: prefill β5.2% from the residual MMAs, a verify
cycle +5.5% from 7.85 instead of 7.5 GB per token, and 2.226 instead of 2.321
tokens per verify on these continuations. These are medians of three run means
across 32 frontiers in three fresh ds4-bench processes per release on one
DGX Spark / GB10: one warm session, 2,048-token incremental prefills and 128
forced greedy tokens per frontier. Clock range 300β2200 MHz; busy clocks
2,177β2,190 MHz.
The R13 release recorded 1,163.5 / 35.08 in the
2Kβ64K sweep after the prefill PR
(3dd964ba, #88).
Incremental prefill falls 7.1% from 4K (1,238.0) to 64K (1,149.9); the 2K point
is each process's first prefill. MTP decode follows draft acceptance on the
generated text (28.7β44.4 per frontier, 2.32 tokens per verify); a verify
cycle averages 66.2 ms. The sweep at c5384b5e
recorded 1,042.2 / 35.12 (2.34 tokens per verify): #88 changed prefill only.
The earlier October 17 sweep
(60d0b689) recorded 1,022.6 / 24.82 with MTP off; 512-token decodes at 2K
with MTP off measure 25.9 tok/s since c5384b5e. A whole 8K prompt (7,936
tokens) prefills at 1,343.8 tok/s (two fresh runs, MTP off). The
GLM VQ page records the optimization steps, numerical contract
and quality screens. Throughput does not
qualify serving limits or long-context quality.
Ling-3.1-flash VQ
Ling-3.1-flash with GVQ1 routed experts and Q4_K/FP8 nonexperts, v2 trunk
(Q4_K KDA decay/gate, Q8_0 embeddings;
model card) recorded
774.1 prefill / 17.70 decode with MTP off across the 32 frontiers of the
2Kβ64K v2 sweep
(709cf8df), and 861.0 / 23.58 with two embedded MTP drafts over 2Kβ32K
(MTP off on that range: 861.7 / 17.83). These are medians of three run means
in three fresh ds4-bench processes per variant on one DGX Spark / GB10: one
warm session, 2,048-token incremental prefills and 128 forced greedy tokens
per frontier. MTP stops at 32K, as in the earlier sweeps. Clock range
300β2200 MHz.
Decode gains 16% at 2K and 20% at 64K over v1: Q4_K decay/gate remove
1.85 GB of each token's 12.6 GB of weight reads, and FP8 latent rows shrink
the part that grows with context. Prefill is unchanged within the run spread;
incremental prefill falls 35% from 4K (964.9) to 64K (625.8) as the nine MLA
layers re-expand the latent cache in each step. MTP decode follows draft
acceptance (20.4β30.1 per frontier). The
prefill-round sweep
(a64e323b, v1, #93) recorded 778.8 / 14.86 and 859.8 / 21.15; the
first sweep
(4d6f096b, #92) 669.6 / 15.24 and 751.6 / 20.99. The
Ling-3.1 page records the execution paths, serving
shapes and quality screens. Throughput does not qualify serving limits or
long-context quality.
Beam-1.0 VQ
Beam-1.0-501B-A23B with GVQ1 routed experts (13-bit on layers 3β10, 12-bit
above) and Q4_K nonexperts
(model card)
recorded 1,241.2 prefill / 25.23 decode across the 32 frontiers of the
2Kβ64K sweep
(1,326.0 / 26.03 to 32K), medians of three run means in three fresh
ds4-bench processes on one DGX Spark / GB10, after the
October 21 optimization round (830.4 /
15.86 before it). Decode
falls from 26.61 at 2K to 23.67 at 64K; prefill peaks at 1,413.7 at 4K and
ends at 1,087.2. With the DFlash2 drafter (draft 7, margin 2) the same
raw-prose sweep runs 1,225.7 / 24.73; five greedy chat prompts gain 22%
(26.23 β 32.03 tok/s). The Beam page records the execution
paths, the Harmony protocol, DFlash2 and the quality gates. Throughput does
not qualify serving limits or long-context quality.
DeepSeek V4.1
DeepSeek V4.1 Flash vq8sh14-q4k-mtpnative, original Engram tables and
grrb-code_fit_n15360-engine recorded 1,039.65 prefill / 38.17 decode
in the October 13 2Kβ64K campaign.
These are medians of three run means across 32 frontiers, using three fresh
Rust-server processes on one DGX Spark / GB10. Each request prefills its full
prompt with zero KV reuse and generates 128 greedy tokens with automatic
embedded MTP. Clock range remains 300β2200 MHz; request-clock medians were
2,184β2,190 MHz.
The warm path is measured separately, because the cache that holds the sidecar mount and the engram open across requests is invisible to a zero-reuse sweep. The warm-path campaign runs the author's five-case HTTP suite on the archived case bytes and reads warm-prefix TTFT at 190.9 β 15.1 ms on the 54-token prose case, 196.7 β 17.3 and 194.1 β 14.1 ms on the other short cases, and 265.7 β 99.1 and 273.4 β 99.5 ms on the two long ones (medians; 45 samples over three passes against the archived pre-change pass). Cold first-touch TTFT improves as well (0.657 β 0.427 s on the same prose case) and decode stays at parity, so the warm cell is not bought with a slower cold path or slower decode.
JLFN ported DeepSeek V4.1 Flash serving to
ds4-dfm-rs (#80), using the
YoungAi engine and
VQ model artifacts.
Curves show per-frontier medians; bands retain the observed minβmax. The
Prefill and Decode tables
list all 96 samples and actual input counts. The 64K frontier has
66,608 actual input tokens and records
969.5 prefill / 39.8 decode at the median.
The graph follows the Qwen card's presentation, but Qwen uses incremental prefill and this sweep uses full prompt prefill. They are separate workloads. The earlier review qualification retains source parity, serving gates and the supplied 40K screenshot comparison. This campaign establishes throughput on the recorded serial corpus, without extending broad long-context quality, 1M throughput or concurrent-bank claims.
Bonsai
Prism Bonsai 2 27B PQ2_0's September 30 prefill evidence uses an RTX 4070 SUPER, not GB10. Three fresh-server pairs on a 2,140-token prompt with 64 outputs record mean prefill 1,023.3; the same tiled runs decode at 18.00 / 17.50 / 18.00. The README reports that observed decode range. Loaded clocks were 2775β2790 MHz. IDs matched the control.
The later split-K report measures 15K/30K workloads. Its decode figures are not combined with this earlier prefill row.
Measuring and qualifying changes
- ds4-perf: inspection, profiling, paired comparisons and serving workloads.
- Optimization playbook: bottlenecks and numerical contracts.
- Speed benchmarks: manual sweeps and plot generation.
- Contributing: fresh processes, matched settings and required proofs.
Use unprofiled timing for throughput; use profiler captures to explain the execution path. Preserve the exact model, quant, prompt, cache state, context, output, bank count and clock conditions. A kernel speedup alone does not establish end-to-end improvement.
Release qualification remains in the release ledgers and migration evidence. The original split matrix, Qwen ABBA/soak, RC.3 serving checks, GLM smoke and K2 admission gates have their own commits and workload limits. Historical PASS results do not qualify a different artifact, a larger prompt or the current machine's live service.








