Upload FINDINGS.md with huggingface_hub

FINDINGS.md

@@ -531,3 +531,57 @@ Cloudflare sits in front of the pod proxy and returns `error code: 524`. A cold
 prefill longer than that cannot complete in one call — measured at ~40k on
 Kimi and ~232k on Qwen. Chunk the prompt (each request extends the cached
 prefix) or expose a TCP port instead.
+
+---
+
+# Session 4 — speculative decoding, and why the regime decides everything
+
+n-gram speculative decoding on Qwen3-Coder-30B-A3B-AWQ, vLLM 0.27.1, 1x A40,
+`--speculative-config '{"method":"ngram","num_speculative_tokens":5,...}'`.
+Same model, same hardware, same quant, two pods running side by side so the
+comparison is simultaneous rather than sequential:
+
+```
+A. pure generation (no prompt/output overlap)
+     baseline    102.21 tok/s
+     ngram        58.49 tok/s      -43%
+
+B. code editing (output re-emits the input)
+     baseline    130.16 tok/s
+     ngram       242.39 tok/s      +86%
+```
+
+**Speculation is not a free win — it is a bet on repetition.** With nothing to
+guess, every draft is rejected and the verification cost is pure loss; that is
+the -43%. Claude Code lives almost entirely in regime B (read a file, emit a
+modified version, repeat identifiers), so it is the right default *there* and
+the wrong default for prose.
+
+vLLM's telemetry, aggregated over both regimes:
+
+```
+drafts 207   draft tokens 1035   accepted 706  = 68.2%
+accepted per position: 179 / 156 / 131 / 120 / 120
+mean accepted length 3.41  ->  ~4.4 tokens emitted per forward pass
+```
+
+This is the measurement llama.cpp never produced: DSpark there exposed no
+`draft_n` at all and moved throughput by 0.0%. The mechanism was never broken —
+the engine was.
+
+## The number that matters for the original goal
+
+Single stream, code editing, one A40 at $0.44/h:
+
+```
+242.39 tok/s  ->  $0.50 per 1M output tokens
+```
+
+The under-$1/1M target is met **on a single stream**, without needing
+concurrency to amortise anything. For reference the same target on Kimi-K3
+required 611 tok/s aggregate against 14.5 measured.
+
+Note also that the baseline itself reads higher here (102-130 tok/s) than the
+77.6 measured earlier: that earlier figure was a 128-token request whose wall
+clock was dominated by per-request overhead. Longer outputs amortise it. Quote
+77.6 for short replies and ~130 for sustained generation.