Add pre-packed NVFP4 decoders and update docs

README.md

@@ -30,10 +30,12 @@ original VAE, so no DiT or encoder changes are needed.
 
 ## Two tiers
 
-| Tier | Folder | Decoder blocks | Resident weights after packing | Use it when |
+| Tier | Folder | Decoder blocks | Decoder on the GPU | Use it when |
 |---|---|---:|---:|---|
-| Quality, **C-v4** | `quality-36L/` | 36 (full depth) | ~1.36 GB | you want output that looks the same as the original decoder |
-| Fast, **F8-v2** | `fast-8L/` | 8 | ~0.30 GB | decode time or memory matters most and you can accept some loss |
+| Quality, **C-v4** | `quality-36L/` | 36 (full depth) | 1.29 GiB | you want output that looks the same as the original decoder |
+| Fast, **F8-v2** | `fast-8L/` | 8 | 0.31 GiB | decode time or memory matters most and you can accept some loss |
+
+Each tier ships in two formats that decode **bit-identically**; see [Files](#files).
 
 For comparison, the original decoder's weights are 9.7 GB in fp32 or 4.8 GB in bf16.
 
@@ -47,8 +49,8 @@ LPIPS (AlexNet) against the original fp32 decoder on the same latents. Lower is
 | FastH3 OmniRef generations (20 clips) | 0.0039 / 46.0 dB | 0.0276 / 36.1 dB |
 | Real videos, VAE-encoded, held out (6 clips) | 0.0038 / 45.0 dB | 0.0330 / 34.3 dB |
 
-These numbers were reproduced from the released files through FastVideo's loader path,
-with weights packed to FP4 and the fused kernels.
+These numbers were reproduced from both released formats through FastVideo's loader,
+with the fused kernels. The two formats produced bit-identical frames on all 35 clips.
 
 ### Blind side-by-side votes
 
@@ -86,8 +88,9 @@ through FlashInfer. The two produce bit-identical output. The fused decoder kern
 were benchmarked on GB200 and DGX Spark.
 
 On GPUs without FP4 tensor cores (Hopper, Ada, Ampere and older) or on CPU/MPS, the
-FP4 kernels are unavailable, so loading or decoding with these files fails. There is
-no automatic fallback. Use the original H3 VAE decoder on that hardware.
+FP4 kernels are unavailable, so decoding with either format fails. There is no
+automatic fallback. Use the original H3 VAE decoder on that hardware. The bf16 file
+is still useful there as a source for other quantization schemes.
 
 ## How to use with FastVideo
 
@@ -98,28 +101,34 @@ no automatic fallback. Use the original H3 VAE decoder on that hardware.
 > Names and formats may change before the code is merged.
 
 1. Install FastVideo from the fork branch, with FlashInfer, on a Blackwell GPU.
-2. Download a tier and convert it to the `.pt` format the loader reads. The
-   conversion only regroups the tensors; values are unchanged.
+2. Download one file. The packed file is the usual choice:
 
    ```bash
-   hf download FastVideo/FastH3-NVFP4-VAE-Decoder --local-dir fasth3-nvfp4-vae
-   python fasth3-nvfp4-vae/convert_to_fastvideo_pt.py \
-       fasth3-nvfp4-vae/quality-36L/decoder.safetensors quality-36L.pt
+   hf download FastVideo/FastH3-NVFP4-VAE-Decoder quality-36L/decoder-nvfp4.safetensors \
+       --local-dir fasth3-nvfp4-vae
    ```
 
-3. Point FastVideo at it and run any MiniMax-H3 / FastH3 pipeline as usual:
+3. Point FastVideo at the file and run any MiniMax-H3 / FastH3 pipeline as usual:
 
    ```bash
-   export FASTVIDEO_H3_VAE_NVFP4_DECODER=$PWD/quality-36L.pt
+   export FASTVIDEO_H3_VAE_NVFP4_DECODER=$PWD/fasth3-nvfp4-vae/quality-36L/decoder-nvfp4.safetensors
    ```
 
+   The variable accepts either format, `decoder-nvfp4.safetensors` or
+   `decoder.safetensors`, directly.
+
 When the VAE loads, FastVideo:
 
-- replaces the decoder with the NVFP4 one;
+- swaps the decoder on the CPU before the VAE moves to the GPU, so the dense
+  decoder it replaces never reaches the GPU;
+- uses the packed FP4 weights as they are, or packs the bf16 weights at load;
 - skips the INT8 ConvRot decoder overlay;
-- packs the weights to FP4, so only the packed weights stay on the GPU;
 - runs the decode in bf16 autocast, the precision the decoders were trained in.
 
+`convert_to_fastvideo_pt.py` is only needed with older checkouts of the branch,
+whose loader read `.pt` files only. It converts `decoder.safetensors`; the packed
+format needs the current loader.
+
 Which host VAE each tier needs:
 
 - **Quality tier (36 blocks):** the stack's `vae/` folder must hold the full
@@ -139,21 +148,63 @@ Useful environment variables:
 ## Files
 
 ```
-quality-36L/decoder.safetensors   # C-v4 weights
-quality-36L/config.json           # architecture + NVFP4 settings
-fast-8L/decoder.safetensors       # F8-v2 weights
+quality-36L/decoder-nvfp4.safetensors   # C-v4, pre-packed NVFP4 (1.39 GB)
+quality-36L/decoder.safetensors         # C-v4, bf16 master weights (4.86 GB)
+quality-36L/config.json                 # architecture, NVFP4 settings, both formats
+fast-8L/decoder-nvfp4.safetensors       # F8-v2, pre-packed NVFP4 (0.33 GB)
+fast-8L/decoder.safetensors             # F8-v2, bf16 master weights (1.10 GB)
 fast-8L/config.json
-convert_to_fastvideo_pt.py        # safetensors -> FastVideo loader .pt
+convert_to_fastvideo_pt.py              # decoder.safetensors -> .pt, for older loaders
 ```
 
-Each `decoder.safetensors` holds `decoder.*` and `post_quant_conv.*` tensors. The
-safetensors header carries the NVFP4 settings (`format`, `nvfp4_metadata`). The
-quantized linear weights are stored as bf16 master weights. FastVideo turns them
-into NVFP4 at load with the same deterministic quantizer used in training, so the
-files on disk are larger than the resident sizes above:
-
-- quality tier: 4.86 GB on disk;
-- fast tier: 1.10 GB on disk.
+### Which format to use
+
+- **`decoder-nvfp4.safetensors` (packed):** pick this to run the model. It is the
+  smallest download and uses the least GPU memory while loading. It contains
+  Blackwell FP4 data, so it is Blackwell-only.
+- **`decoder.safetensors` (bf16):** this is the reference copy. Use it to
+  re-quantize, fine-tune, run on a different FP4 kernel stack, or check the packed
+  file. FastVideo packs it at load with the same deterministic quantizer used in
+  training.
+
+Peak memory while FastVideo loads the VAE (GB200; the encoder and the rest of the
+VAE are included):
+
+| | GPU, packed | GPU, bf16 | Host RAM (both formats) |
+|---|---:|---:|---:|
+| Quality tier, on the full 36-block VAE | 1.97 GiB | 9.78 GiB | about 23 GiB |
+| Fast tier, on the 26-block light VAE | 0.99 GiB | 2.78 GiB | about 14 GiB |
+
+The GPU memory after loading is the same for both formats. Host RAM peaks the same
+way for both because the host VAE's own fp32 checkpoint is read in full.
+
+### Layout
+
+Both formats store tensors under `decoder.*` and `post_quant_conv.*`. The
+safetensors header carries `format` and `nvfp4_metadata`, a JSON string with the
+block size, activation scale, number of NVFP4 linears and decoder depth.
+
+- **`decoder.safetensors`**
+  - `format`: `fastvideo_h3_decoder_nvfp4_deploy_v1`.
+  - The quantized linears' weights and biases are bf16.
+  - Every other tensor is fp32, the dtype it has in the trained decoder.
+- **`decoder-nvfp4.safetensors`**
+  - `format`: `fastvideo_h3_decoder_nvfp4_packed_v1`.
+  - `nvfp4_metadata` also describes the layout (`packed_layout`) and the GPU and
+    FlashInfer version used to pack.
+  - Each quantized linear `<name>` has these tensors:
+
+    | Tensor | Contents |
+    |---|---|
+    | `<name>.packed_weight` | uint8 `[out, in/2]`, two E2M1 codes per byte (FlashInfer `fp4_quantize` order) |
+    | `<name>.weight_inv_scale` | per-16-element E4M3 block scales, stored as uint8, in FlashInfer's swizzled `SfLayout.layout_128x4` |
+    | `<name>.weight_global_sf` | fp32 scalar |
+    | `<name>.bias` | bf16 |
+    | `<name>.input_amax` | fp32 static activation range, unused with the unit activation scale these decoders use |
+
+  - Norms, `proj_in`, `proj_out`, register tokens and `post_quant_conv` stay
+    fp32. That is the dtype the decoder was trained and validated with, so the
+    output stays bit-identical.
 
 ## Training