File size: 1,079 Bytes
210896d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | # Methods and measurement notes
## Why one-step error and not video PSNR?
H3's sampler is chaotic: computing only the AdaLN in fp32 instead of bf16 already moves a 20-step clip to
~17 dB PSNR against the shipped model. End-of-run pixel metrics therefore cannot rank recipes. Each recipe is
instead fed the exact recorded inputs of every step and compared with the BF16 output of that step.
## Measurement notes
- **Noise floor ~1–2%.** Replays are bitwise exact within one container but differ by 1–2% across GPU hosts, and
any bf16-level change lands at that level. Compare recipes within a run; each run reports its own floor.
- **Size** is the approximate transformer checkpoint size with the rank-12 AdaLN (NVFP4 counted as 4.5 bits per
weight including block scales).
- **Simulation vs ComfyUI.** NVFP4 activation rounding is implementation-defined: ComfyUI's CUDA and Triton kernels
agree with each other but differ from ModelOpt's simulation by ~2% on a quantized activation. Rankings should
carry over; final quality is judged in ComfyUI itself (below).
|