METHODOLOGY: rotation A/B as prose explanation, drop the table
Browse files- METHODOLOGY.md +1 -10
METHODOLOGY.md
CHANGED
|
@@ -14,16 +14,7 @@ We also deliberately ship it as **one self-contained file** (not a pack + extern
|
|
| 14 |
|
| 15 |
Per DiT block linear (`qkv_proj`, `out_proj`, `fc1`, `fc2`):
|
| 16 |
|
| 17 |
-
1. **QuaRot-style block-256 Hadamard rotation.** Multiplying the weight (and, at runtime, the activation) by a block-256 Hadamard matrix spreads per-channel outliers so a 4-bit grid captures the signal instead of clipping the spikes.
|
| 18 |
-
|
| 19 |
-
**Measured β same-weights A/B** (identical weights, shell, AdaLN, scene, seed, sampler and step count; the *only* difference is rotation on vs off; 832Γ480, 61f, `euler` 4-step, seed `757358688076805`, dense):
|
| 20 |
-
|
| 21 |
-
| block mode | lapvar (sharpness) | CLIP-IQA β | MUSIQ β | s/step |
|
| 22 |
-
|---|---|---|---|---|
|
| 23 |
-
| **rotated (shipped)** | **377.5** | **0.423** | **44.6** | 2.16 |
|
| 24 |
-
| un-rotated (plain NVFP4) | 220.9 | 0.386 | 39.4 | 1.83 |
|
| 25 |
-
|
| 26 |
-
Rotation wins **every** quality metric β lapvar **+71%**, CLIP-IQA +0.037, MUSIQ +5.2 β and visibly: the un-rotated render carries a milky rain-bloom haze that softens mid-ground detail and neon glyphs. Both variants are bit-deterministic across repeats (SSIM 1.0), so the gap is real, not seed noise. **Cost:** the runtime Hadamard adds **~15%/step** and gives a *different* draw for a given seed (it reorders which weights land on the fp4 grid). **Honest nuance:** un-rotated NVFP4 is *not* broken β at enough steps on easy/low-contrast scenes it looks fine (many community "stock nvfp4" H3 files do) β but in a like-for-like isolation rotation is the sharper half, which is why we ship it. And despite the rotation cost, our full path is at **parity** with a community stock-nvfp4 file on the same card (10.51 vs 10.37 s/step @768p/61f, +1.4% noise β nunchaku's W4A4 kernel is efficient and high-res is attention-bound), while running **~1.9Γ faster wall-clock** via the 4-step distill.
|
| 27 |
2. **NVFP4** β fp4 (e2m1) packed 2-per-byte + a **group-16 fp8-e4m3 microscale** (336 group-scales per channel on the big linears). The fine per-group scale is what preserves detail; a coarse 1-scale-per-channel int4 (~2.3Γ the quant error on outlier-heavy weights) does not.
|
| 28 |
3. **bf16 islands** kept unquantized: patch/time embed, AdaLN/modulation, token_refiner, proj_in/out. Small, sensitive, not worth quantizing.
|
| 29 |
|
|
|
|
| 14 |
|
| 15 |
Per DiT block linear (`qkv_proj`, `out_proj`, `fc1`, `fc2`):
|
| 16 |
|
| 17 |
+
1. **QuaRot-style block-256 Hadamard rotation.** Multiplying the weight (and, at runtime, the activation) by a block-256 Hadamard matrix spreads per-channel outliers so a 4-bit grid captures the signal instead of clipping the spikes. We confirmed it in a clean same-weights A/B (identical weights and settings, only rotation toggled): rotation wins every no-reference IQA metric and lifts a Laplacian-variance sharpness proxy ~70%, and it shows β the un-rotated render carries a milky bloom that softens mid-ground detail and neon glyphs, while rotated keeps crisp edges. Both runs are bit-deterministic, so the gap is real. The cost is ~15%/step and a different per-seed draw. Honest nuance: un-rotated NVFP4 is *not* broken β at enough steps on easy scenes it looks fine (many community "stock nvfp4" H3 files do) β but like-for-like, rotation is the sharper half, which is why we ship it. And despite the rotation cost, our full path is at parity with a community stock-nvfp4 file on the same card (+1.4%/step, noise) while running ~1.9Γ faster wall-clock via the 4-step distill.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
2. **NVFP4** β fp4 (e2m1) packed 2-per-byte + a **group-16 fp8-e4m3 microscale** (336 group-scales per channel on the big linears). The fine per-group scale is what preserves detail; a coarse 1-scale-per-channel int4 (~2.3Γ the quant error on outlier-heavy weights) does not.
|
| 19 |
3. **bf16 islands** kept unquantized: patch/time embed, AdaLN/modulation, token_refiner, proj_in/out. Small, sensitive, not worth quantizing.
|
| 20 |
|