reports/reference_consolidation.md

InferScale Research Consolidation

Results in this report are simulator outputs unless an external-measurement validation section is present. Public device/model timings use analytical reference profiles rather than measured hardware benchmarks.

Study protocol

• Matched workload seeds: 12
• Bootstrap resamples: 600
• Robust Pareto objectives: p95 step TTFT, unused speculative prefetch, and mean prefix-HBM residency.
• Offline reference: bounded full-trace serving oracle over the declared policy/action-plan family.

Robust policy ranking

Robust winner: Top-1 decayed

Nominal first-seed winner: Top-1 decayed

Rank	Policy	Median p95 TTFT	95% CI of mean	TTFT wins	Pareto stability	Median oracle regret	Worst seed
1	Top-1 decayed	6,474.9 ms	5,105.7-8,966.0 ms	41.7%	75.0%	59.3 ms	14,688.1 ms
2	No prefetch	6,490.7 ms	5,079.3-8,648.9 ms	41.7%	91.7%	34.7 ms	14,328.2 ms
3	Utility-aware multi-step	6,507.5 ms	5,199.5-9,163.0 ms	8.3%	58.3%	133.1 ms	14,634.7 ms
4	Multi-step top-k	6,640.3 ms	5,341.7-9,082.2 ms	8.3%	83.3%	215.0 ms	15,000.0 ms

Offline constrained oracle

• Candidate plans per seed: 19
• Median oracle p95 TTFT: 6,408.6 ms
• Definition: bounded-full-trace-serving-oracle

The oracle exhaustively selects among the evaluated deployable policies plus clairvoyant future-set plans over H=1..5 and K=1..3 on each complete trace, under the same cache/bandwidth constraints. It is a bounded information upper bound, not a proof of globally optimal action scheduling.

Interpretation guardrails

1. Bootstrap intervals quantify variation across simulated workload seeds; they do not quantify real-hardware model error.
2. The offline oracle is bounded to an explicit action-plan family and should not be described as a globally optimal serving controller.
3. Analytical reference profiles are hypotheses about timing. Hardware claims require independent measured validation.
4. External calibration can correct global scale bias but does not prove fidelity for unseen models, devices, or scheduler regimes.