pottokao commited on
Commit
4e8b11d
Β·
verified Β·
1 Parent(s): 8140ab9

METHODOLOGY: rotation A/B as prose explanation, drop the table

Browse files
Files changed (1) hide show
  1. METHODOLOGY.md +1 -10
METHODOLOGY.md CHANGED
@@ -14,16 +14,7 @@ We also deliberately ship it as **one self-contained file** (not a pack + extern
14
 
15
  Per DiT block linear (`qkv_proj`, `out_proj`, `fc1`, `fc2`):
16
 
17
- 1. **QuaRot-style block-256 Hadamard rotation.** Multiplying the weight (and, at runtime, the activation) by a block-256 Hadamard matrix spreads per-channel outliers so a 4-bit grid captures the signal instead of clipping the spikes.
18
-
19
- **Measured β€” same-weights A/B** (identical weights, shell, AdaLN, scene, seed, sampler and step count; the *only* difference is rotation on vs off; 832Γ—480, 61f, `euler` 4-step, seed `757358688076805`, dense):
20
-
21
- | block mode | lapvar (sharpness) | CLIP-IQA ↑ | MUSIQ ↑ | s/step |
22
- |---|---|---|---|---|
23
- | **rotated (shipped)** | **377.5** | **0.423** | **44.6** | 2.16 |
24
- | un-rotated (plain NVFP4) | 220.9 | 0.386 | 39.4 | 1.83 |
25
-
26
- Rotation wins **every** quality metric β€” lapvar **+71%**, CLIP-IQA +0.037, MUSIQ +5.2 β€” and visibly: the un-rotated render carries a milky rain-bloom haze that softens mid-ground detail and neon glyphs. Both variants are bit-deterministic across repeats (SSIM 1.0), so the gap is real, not seed noise. **Cost:** the runtime Hadamard adds **~15%/step** and gives a *different* draw for a given seed (it reorders which weights land on the fp4 grid). **Honest nuance:** un-rotated NVFP4 is *not* broken β€” at enough steps on easy/low-contrast scenes it looks fine (many community "stock nvfp4" H3 files do) β€” but in a like-for-like isolation rotation is the sharper half, which is why we ship it. And despite the rotation cost, our full path is at **parity** with a community stock-nvfp4 file on the same card (10.51 vs 10.37 s/step @768p/61f, +1.4% noise β€” nunchaku's W4A4 kernel is efficient and high-res is attention-bound), while running **~1.9Γ— faster wall-clock** via the 4-step distill.
27
  2. **NVFP4** β€” fp4 (e2m1) packed 2-per-byte + a **group-16 fp8-e4m3 microscale** (336 group-scales per channel on the big linears). The fine per-group scale is what preserves detail; a coarse 1-scale-per-channel int4 (~2.3Γ— the quant error on outlier-heavy weights) does not.
28
  3. **bf16 islands** kept unquantized: patch/time embed, AdaLN/modulation, token_refiner, proj_in/out. Small, sensitive, not worth quantizing.
29
 
 
14
 
15
  Per DiT block linear (`qkv_proj`, `out_proj`, `fc1`, `fc2`):
16
 
17
+ 1. **QuaRot-style block-256 Hadamard rotation.** Multiplying the weight (and, at runtime, the activation) by a block-256 Hadamard matrix spreads per-channel outliers so a 4-bit grid captures the signal instead of clipping the spikes. We confirmed it in a clean same-weights A/B (identical weights and settings, only rotation toggled): rotation wins every no-reference IQA metric and lifts a Laplacian-variance sharpness proxy ~70%, and it shows β€” the un-rotated render carries a milky bloom that softens mid-ground detail and neon glyphs, while rotated keeps crisp edges. Both runs are bit-deterministic, so the gap is real. The cost is ~15%/step and a different per-seed draw. Honest nuance: un-rotated NVFP4 is *not* broken β€” at enough steps on easy scenes it looks fine (many community "stock nvfp4" H3 files do) β€” but like-for-like, rotation is the sharper half, which is why we ship it. And despite the rotation cost, our full path is at parity with a community stock-nvfp4 file on the same card (+1.4%/step, noise) while running ~1.9Γ— faster wall-clock via the 4-step distill.
 
 
 
 
 
 
 
 
 
18
  2. **NVFP4** β€” fp4 (e2m1) packed 2-per-byte + a **group-16 fp8-e4m3 microscale** (336 group-scales per channel on the big linears). The fine per-group scale is what preserves detail; a coarse 1-scale-per-channel int4 (~2.3Γ— the quant error on outlier-heavy weights) does not.
19
  3. **bf16 islands** kept unquantized: patch/time embed, AdaLN/modulation, token_refiner, proj_in/out. Small, sensitive, not worth quantizing.
20