shared_material/performance.md

# Shared-material validation, 2026-09-22

Hardware: RTX PRO 6000 Blackwell 96 GB. Same pinned vLLM image as the base runtime,
88 original BF16 output rows, mixed NVFP4/FP8 backbone, PLE host offload,
eager execution, 4096 context, max_num_seqs=4. Cache grid resolved to 800 tokens.

The three frozen synthetic materials were a short record and two long records
with contradictory color, quantity and train departure facts. Each had six
questions and three options per question. Long records included 80 irrelevant
archive entries. Actual prefix size was 2195 tokens plus 205 explicit newline
padding tokens; short size was 195 plus 605. Token sequences were identical
between the recompute and shared executions. Reference requests each used a
unique cache salt and every reference candidate reported zero cached tokens.

## Computation evidence

For either long material and all 18 candidates:

- Fully recomputed candidate input: 44,166 token positions.
- Reused material prefix: 18 × 2400 = 43,200 token positions, reported by the engine.
- Uncached candidate suffixes: 966 token positions.
- Hot batch warmup: 2401 input tokens, 2400 cached, 1 computed.
- Cold preparation: 2401 input tokens, 0 cached, 2401 computed, followed by the
  hot batch warmup above.

Thus candidate-only prefill work falls by 97.81%. Counting preparation/warmup,
cold work is 3368 versus 44166 token positions (92.37% less), and resident work
is 967 (97.81% less). These are token-work measurements, not equivalent reductions
in GPU FLOPs, wall time, energy or billing. All 486 shared candidate responses
in the primary test reported the whole material prefix cached.

## Accuracy and variation

| Primary test mode | Correct / question evaluations |
|---|---:|
| Full recomputation | 53/54 |
| Cold shared serial | 54/54 |
| Hot shared serial | 52/54 |
| Hot shared 4-way | 54/54 |

There are only 18 distinct questions over three materials; repetitions and modes
are not independent new examples. The three wrong responses all concern the
same `9 + 4` total. Comparing each shared response to its matched recompute
response gives 157/162 matching decisions; a maximum independent Yes-probability
difference of 0.542 was observed. This is **not** a numerical-equivalence claim.
The baseline itself varies, so these differences cannot all be causally
attributed to prefix caching. No engine/precision root cause is asserted.

The first benchmark script's final all-gold assertion failed on a baseline
arithmetic error, after all cache/isolation/recovery checks had completed. The
raw results were retained in full. The reporter was corrected to retain and
report model errors rather than requiring a perfect model or rerunning to select
better results. No fixture, expected answer or inference prompt was changed.

## Additional live checks

- Two opposite materials interleaved A/B/A/B: 24/24 correct, full prefix hits.
- Engine cache reset then existing material ID reuse: 6/6 correct; warmup recorded
  zero cached tokens and all following candidates reused the rebuilt prefix.
- Separate cold parallel test: 36/36 correct, three repeats of each long material.
- New question suffixes appended through existing IDs: 18/18 correct and full
  prefix hits, without resending material. Whole appended batches contained three
  questions / eight candidates, median 533 ms for A and 631 ms for B.
- Final packaged service, Chinese material/questions: 2/2 correct, six candidates
  reused all 800 common prefix tokens.
- CPU contract tests: 6 passed (validation, LRU/TTL/delete, exact prefix assembly,
  missing cache-evidence rejection, all-batch context preflight, ephemeral cleanup).
- Re-extracting `no`/`yes` rows using the packaged extractor reproduced the
  published optional head byte-for-byte; the original 86 rows are unchanged.

The service was restored to the user's prior 0.8B GPU workload after measurement.
No official JevBench score, model quality improvement or leaderboard claim follows
from these cache and synthetic workflow tests.