byte-vortex commited on
Commit
a8def04
·
verified ·
1 Parent(s): 6d1dd5a

Update pages/executive-summary/page.md

Browse files
Files changed (1) hide show
  1. pages/executive-summary/page.md +0 -56
pages/executive-summary/page.md CHANGED
@@ -55,59 +55,3 @@ All empirical claims were run as real, independently-verifiable Hugging Face Job
55
  </div>
56
 
57
  ````
58
-
59
-
60
- ---
61
- <!-- trackio-cell
62
- {"type": "markdown", "id": "cell_5fc6b6dfa861", "created_at": "2026-08-01T19:06:23+00:00", "title": "Executive summary", "pinned": true, "pinned_at": "2026-08-01T19:06:23+00:00"}
63
- -->
64
- Reproduction of "A Random Matrix Theory Perspective on the Consistency of Diffusion Models" (arXiv 2602.02908), an ICML 2026 ORAL presentation (Wang, Zavatone-Veth, Pehlevan, Harvard University).
65
-
66
- All five core claims genuinely verified, plus a scoped-down but methodologically faithful attempt at the paper's sixth, most compute-heavy claim:
67
-
68
- - Claim 1 (linear theory predicts cross-split consistency): confirmed on REAL MNIST data -- cross-split generated samples were 7.4x more similar to each other than to their nearest training neighbor, using the exact closed-form linear denoiser from Section 2.
69
- - Claim 2 (self-consistency renormalization equation, Result 4.1): confirmed to 2.6% max relative error via Monte Carlo simulation, and independently cross-validated against the authors' own officially released code (github.com/Animadversio/diffusion-consistency-rmt) -- our fixed-point solver was confirmed algebraically identical to their solve_kappa implementation.
70
- - Claim 3 (factorized variance law, Result 4.2, Eq. 7): confirmed to 19.5% max relative error using the paper's exact stated formula (anisotropy x inhomogeneity x scaling factorization), verified against Monte Carlo simulation.
71
- - Claim 4 (fractional matrix power extension to full sampling trajectories, Results 5.1 & 5.2, Section 5): confirmed by independently implementing the paper's two stated closed-form integrals -- the deterministic equivalent of E[Sigma_hat^(1/2)] (Eq. 9, max 0.7% relative error) and of Var[v^T Sigma_hat^(1/2) x_bar] (Eq. 10, max 11.5% relative error) -- against direct Monte Carlo simulation across n in {60, 100, 200, 400}. No official example code was published for this result yet, so this quadrature scheme is our own, built directly from the paper's stated formulas.
72
- - Claim 5 (deep network validation): tested at two scales. A small MLP on real MNIST first confirmed the qualitative consistency pattern (2.2x cross-split similarity vs. nearest-neighbor). A more thorough follow-up trained a real convolutional UNet on real CIFAR10 across four dataset sizes (n=100 to 20,000) and tested all three of the paper's stated predictions directly: (a) UNet cross-split distance stayed within a bounded ratio (1.2x-2.5x) of a genuine closed-form linear-diffusion baseline run through the identical reverse process; (b) an overshrinkage signature was present at every scale -- low-variance covariance eigenmodes were shrunk substantially more than high-variance ones (correlation 0.78-0.89); (c) cross-split deviation was eigenmode-dependent (anisotropic) and decayed as n grew. 4/4 sub-checks were consistent with the paper's predictions.
73
-
74
- All empirical claims were run as real, independently-verifiable Hugging Face Jobs with public execution logs, not just local/offline runs.
75
-
76
- ## Scope & cost
77
- | | This reproduction | Full replication |
78
- |---|---|---|
79
- | Scope | All 5 claims verified, including a toy-scale but methodologically complete deep-network validation (consistency, overshrinkage, anisotropy) | Same claims at the paper's full scale: 7 datasets, UNet + DiT, 50k training steps per run (~100+ total runs) |
80
- | Hardware | Hugging Face Jobs (cpu-basic for Claims 1-4; a10g-small for Claim 5's UNet training) | Comparable for Claims 1-4; Claim 5 at full scale needs substantially more GPU time |
81
- | Compute time | ~35-50 min for Claims 1-4; the UNet sweep across 4 dataset sizes added roughly another hour on a10g-small | Similar for Claims 1-4; full-scale Claim 5 is the dominant cost |
82
- | Cost | $0 (free-tier Jobs credit) | Minimal for Claims 1-4; full-scale Claim 5 needs meaningfully more compute budget |
83
- | Outcome | 5/5 claims verified with quantified error bounds where applicable; Claim 2 additionally cross-validated against official code; Claim 4 independently derived since no official example code was published for it; Claim 5's deep-network validation directionally supported across all four tested sub-predictions |
84
-
85
-
86
- ---
87
- <!-- trackio-cell
88
- {"type": "figure", "id": "cell_5ad936739324", "created_at": "2026-08-01T19:06:24+00:00", "title": "Reproduction poster", "pinned": true, "pinned_at": "2026-08-01T19:06:25+00:00"}
89
- -->
90
- ````html
91
- <div style="font-family: -apple-system, sans-serif; max-width: 900px; margin: 0 auto; padding: 24px; border: 1px solid #ddd; border-radius: 8px;">
92
- <h2 style="margin-top:0;">Reproduction Poster</h2>
93
- <h3 style="color:#555; font-weight:normal;">A Random Matrix Perspective on the Consistency of Diffusion Models</h3>
94
- <p style="color:#777;">arXiv 2602.02908 &middot; OpenReview iPjuUQbkfl &middot; ICML 2026 Oral</p>
95
- <table style="width:100%; border-collapse: collapse; margin: 16px 0;">
96
- <tr style="background:#f5f5f5;"><th style="text-align:left; padding:8px; border:1px solid #ddd;">Claim</th><th style="text-align:left; padding:8px; border:1px solid #ddd;">Result</th></tr>
97
- <tr><td style="padding:8px; border:1px solid #ddd;">1. Linear theory predicts cross-split consistency</td><td style="padding:8px; border:1px solid #ddd;">Confirmed: 7.4x closer cross-split than to nearest neighbor (real MNIST)</td></tr>
98
- <tr><td style="padding:8px; border:1px solid #ddd;">2. Self-consistency equation / renormalized noise</td><td style="padding:8px; border:1px solid #ddd;">Confirmed: 2.6% max error, matches authors' released code</td></tr>
99
- <tr><td style="padding:8px; border:1px solid #ddd;">3. Variance factorization: anisotropy &times; inhomogeneity</td><td style="padding:8px; border:1px solid #ddd;">Confirmed: 19.5% max error vs. Monte Carlo</td></tr>
100
- <tr><td style="padding:8px; border:1px solid #ddd;">4. Fractional matrix power / sampling trajectories</td><td style="padding:8px; border:1px solid #ddd;">Confirmed: 0.7% / 11.5% max error vs. Monte Carlo</td></tr>
101
- <tr><td style="padding:8px; border:1px solid #ddd;">5. Deep network validation (MLP/MNIST, then UNet/CIFAR10)</td><td style="padding:8px; border:1px solid #ddd;">Confirmed at both scales; 4/4 sub-checks directionally supported</td></tr>
102
- </table>
103
- <p><strong>Claim 5 summary:</strong> a small MLP on real MNIST first confirmed the qualitative consistency
104
- pattern (2.2x closer cross-split than to nearest neighbor). A more thorough follow-up trained a real
105
- convolutional UNet on real CIFAR10 across dataset sizes n = 100&ndash;20,000, testing all three of the
106
- paper's stated predictions directly: (a) cross-split consistency stayed within a bounded ratio (1.2x&ndash;2.5x)
107
- of a genuine closed-form linear-diffusion baseline; (b) overshrinkage of low-variance eigenmodes relative
108
- to high-variance ones (correlation 0.78&ndash;0.89); (c) eigenmode-dependent (anisotropic) cross-split
109
- deviation that decayed with n. All four sub-checks were consistent with the paper's predictions.</p>
110
- <p style="color:#999; font-size: 0.9em;">Full logbook: byte-vortex/repro-diffusion-consistency</p>
111
- </div>
112
-
113
- ````
 
55
  </div>
56
 
57
  ````