Spaces:
Running
Running
| # InferScale Research Consolidation | |
| > Results in this report are simulator outputs unless an external-measurement validation section is present. | |
| > Public device/model timings use analytical reference profiles rather than measured hardware benchmarks. | |
| ## Study protocol | |
| - Matched workload seeds: **12** | |
| - Bootstrap resamples: **600** | |
| - Robust Pareto objectives: p95 step TTFT, unused speculative prefetch, and mean prefix-HBM residency. | |
| - Offline reference: bounded full-trace serving oracle over the declared policy/action-plan family. | |
| ## Robust policy ranking | |
| **Robust winner:** Top-1 decayed | |
| **Nominal first-seed winner:** Top-1 decayed | |
| | Rank | Policy | Median p95 TTFT | 95% CI of mean | TTFT wins | Pareto stability | Median oracle regret | Worst seed | | |
| |---:|---|---:|---:|---:|---:|---:|---:| | |
| | 1 | Top-1 decayed | 6,474.9 ms | 5,105.7-8,966.0 ms | 41.7% | 75.0% | 59.3 ms | 14,688.1 ms | | |
| | 2 | No prefetch | 6,490.7 ms | 5,079.3-8,648.9 ms | 41.7% | 91.7% | 34.7 ms | 14,328.2 ms | | |
| | 3 | Utility-aware multi-step | 6,507.5 ms | 5,199.5-9,163.0 ms | 8.3% | 58.3% | 133.1 ms | 14,634.7 ms | | |
| | 4 | Multi-step top-k | 6,640.3 ms | 5,341.7-9,082.2 ms | 8.3% | 83.3% | 215.0 ms | 15,000.0 ms | | |
| ## Offline constrained oracle | |
| - Candidate plans per seed: **19** | |
| - Median oracle p95 TTFT: **6,408.6 ms** | |
| - Definition: `bounded-full-trace-serving-oracle` | |
| The oracle exhaustively selects among the evaluated deployable policies plus clairvoyant future-set plans over H=1..5 and K=1..3 on each complete trace, under the same cache/bandwidth constraints. It is a bounded information upper bound, not a proof of globally optimal action scheduling. | |
| ## Interpretation guardrails | |
| 1. Bootstrap intervals quantify variation across simulated workload seeds; they do not quantify real-hardware model error. | |
| 2. The offline oracle is bounded to an explicit action-plan family and should not be described as a globally optimal serving controller. | |
| 3. Analytical reference profiles are hypotheses about timing. Hardware claims require independent measured validation. | |
| 4. External calibration can correct global scale bias but does not prove fidelity for unseen models, devices, or scheduler regimes. | |