Text-to-Image
Diffusion Single File
comfyui

krea2_turbo_nvfp4: full_precision_matrix_mult=true on all layers makes it 2.5x slower (11.6 s vs 4.6 s)

#25
by sjoe1244 - opened

Hi, thanks for the Krea 2 release.

In krea2_turbo_nvfp4.safetensors every one of the 256 quantized layers has "full_precision_matrix_mult": true in _quantization_metadata. ComfyUI reads that and dequantizes the weights to bf16 each step, so the NVFP4 matmul is never used.

On an RTX 5080 (ComfyUI 0.42.3, torch 2.13+cu130, 864x1536, 8 steps, warm): 11.6 s with the file as shipped, 4.6 s with only that flag removed from the header (tensor bytes unchanged). A profile of the shipped file shows 79% of GPU time in bf16 GEMMs and 2048 dequantize_nvfp4 calls. comfy_kitchen.scaled_mm_nvfp4 runs at about 792 TFLOPS on the same card.

Is the flag intentional (for example to keep activations in bf16)? If yes, it may be worth documenting. If not, shipping the file without it makes Turbo about 2.5x faster on Blackwell. I only compared one seed side by side (both clean); no wider quality test.

Details and a 1 KB script: https://huggingface.co/sjoe1244/krea2-turbo-nvfp4-fast-matmul-fix

Sign up or log in to comment