# OmniRef native ComfyUI export

The INT8 file packaged in this repository is an unchanged copy of the existing
Comfy-produced artifact, SHA256
`851b2dc9670ed864b7f1e951ac9da047d83b1c04ded9bfcc1a9e5bba0ee33f80`.
Its source identity and sampled PDD/quantized-row checks are in `../provenance/`.
The converter below produced the BF16 variant. Running the optional quantizer
commands below creates a new PTQ variant; they are not a claim of byte-for-byte
reproduction of the supplied INT8 file, whose reduced AdaLN weights are FP16
rather than this converter's FP32.

Source: `FastVideo/FastVideo-FastH3-Omni-8-Step-V1` at
`88d512c5e7cc38dda499fbb65688ce58611f9c80`, component `transformer_ref`.
The caller must verify the source shards against the frozen source identity
before conversion. The converter checks every indexed tensor and its geometry;
that check alone does not prove the files match the Hub revision.

The target is native ComfyUI's MiniMax H3 architecture, pinned to
[`b1a247c9cf9dbfda4b2c0a0c1d50950667eeaa0c`](https://github.com/Comfy-Org/ComfyUI/tree/b1a247c9cf9dbfda4b2c0a0c1d50950667eeaa0c).
Official model tools are pinned to
[`d6797787e6bdb1a1fb0094d588a26f8e71a1c757`](https://github.com/Comfy-Org/comfy-model-tools/tree/d6797787e6bdb1a1fb0094d588a26f8e71a1c757).
The small pinned architecture, detector, quantizer, source URLs, hashes and GPL
licenses are under `upstream/`. No upstream source tree or local virtualenv is
required in the model repository.

## Transformations

- Rename all 688 source tensors to the consumer's names; unknown/missing keys
  or mismatched geometry abort conversion.
- Fuse Q, K, V in that row order, including the token refiner. Swap the two
  SwiGLU input halves from source value/gate to native Comfy gate/value.
- Preserve all 32 PDD heads. Native Comfy stores the first full head and each
  later head's offset from the first. Compute those offsets in FP32; source
  BF16 values and their differences are exactly representable at this stage.
- Generate the nonpersistent source RoPE frequencies as the persistent buffer
  expected by Comfy, using the original FP32 formula.
- Preserve source dtypes otherwise. Native Comfy casts its input, timestep and
  output projection islands to FP32 when loading.
- `--adaln-curve-rank 0` retains the dense timestep MLP and full AdaLN matrices.
  `--adaln-curve-rank 8 --curve-grid 1025` uses the same kind of time-embedding
  curve compression as the V2 Comfy pruned checkpoint: fit a shared SVD basis,
  project all 51 modulation matrices and replace the timestep MLP by a
  linearly interpolated coordinate table on `[0,1]`. This is an approximation,
  not removal of unused model blocks. The export stores table and reduced
  matrices in FP32; the V2 checkpoint stores reduced matrices in FP16. Its
  receipt records fit residual and 129 independent holdout timesteps on 64
  sampled output rows per projection. No output-quality claim follows from
  these modulation checks alone.

Every source tensor contributes to the output or the curve projection; no
trained transformer blocks or PDD heads are discarded. The four timestep-MLP
tensors are replaced only in the explicit curve variant.

## Run

The conversion requires CPU PyTorch, NumPy and safetensors. It uses bounded
buffers and a scratch directory, instead of loading the 70.1 GB source into
RAM. The output writer hashes bytes as it streams them. A partial file is
renamed only after a complete write; existing outputs and scratch directories
are not overwritten.

```bash
python convert_omniref_comfy.py \
  --src /verified/source/transformer_ref \
  --contract /verified/source/fastvideo_inference.json \
  --output /staging/diffusion_models/fasth3_omniref_8step_pruned_bf16.safetensors \
  --scratch /allocated/scratch/omniref-comfy-curve-r8 \
  --adaln-curve-rank 8 --curve-grid 1025 --threads 4
```

This produces 582 tensors with the curve recipe or 585 without it. The full
source contains 688 tensors and 70,142,174,464 tensor bytes. Source and output
heads are not interchangeable by renaming: the PDD representation differs.

After validating the BF16-format export, run the pinned official quantizer's
`--dry-run`, then quantize on an allocated GPU:

```bash
python upstream/quant_int8_convrot.py \
  /staging/diffusion_models/fasth3_omniref_8step_pruned_bf16.safetensors \
  --dry-run
python upstream/quant_int8_convrot.py \
  /staging/diffusion_models/fasth3_omniref_8step_pruned_bf16.safetensors \
  /staging/diffusion_models/fasth3_omniref_8step_pruned_int8_convrot.safetensors \
  --verify-report /staging/int8-convrot-verification.txt
```

The tool defaults to absmax INT8, a minimum GEMM dimension of 256 and the largest
eligible ConvRot group in 256/64/16. Do not add MSE clipping, FP32 downcasting,
W4A8 or W6A8 when claiming this default recipe. Final/output heads and the
small reduced AdaLN matrices pass through. Execution is CUDA-only in this
version and retains the output dictionary in CPU RAM; allocate enough RAM for
the roughly 22 GB INT8 result plus work buffers. The dry run itself is CPU.

## Validation and inference limits

`python test_conversion.py` runs six CPU regression checks against actual
pinned ComfyUI architecture and detector definitions. Ordinary CPU Linear and
RMSNorm replace CUDA operations for these checks. Tests cover strict state-dict
loading for dense and curve forms, fused QKV order, native SwiGLU output,
native PDD head reconstruction, rejected unexpected tensors and streamed-file
hash/overwrite protection. `audit_native_geometry.py --src ...` additionally
checks every real-source key and shape against dense and rank-8 native model
skeletons on the meta device, without allocating model weights.

These are checkpoint format and CPU arithmetic checks. They do not validate
CUDA quantization kernels or an end-to-end generated clip.

Native Comfy inference differs from the frozen FastVideo source contract:

- Native PDD uses a unit `[1,0]` fine grid; the source's grid maximum is 0.999.
  Native PDD storage is handled by this converter, but its sampler grid is
  consumer behavior and is not changed by model metadata.
- Native Comfy's VSA node uses `(4,4,4)` video cubes, while the source contract
  specifies tile size 128 and reference policy `p2_multi_region`.
- Native attention is dense until a sparse-attention patch is selected.
- Source generation requires 8 forwards, guidance 1, video/audio shifts 12/3
  and the declared 32-interval head partition. Do not copy V2's video shift 10
  or present the V2 text-to-video template as an OmniRef workflow.

The original inference contract is retained verbatim in safetensors metadata
and in the conversion receipt. Do not claim bit-identical inference or reuse
GB200 benchmark numbers for native Comfy without a matched runtime check.

## Other checkpoints

The existing V2 Comfy text encoders and complete original 36L video/audio VAEs
are valid native-Comfy artifacts and can be copied with their original hashes
and license attribution. MiniMax-H3 weights retain the MiniMax-H3 Community
License; the tooling's GPL license is separate from the model license.

The FastVideo FP8 source needs a separate format bridge for fused QKV/scales,
SwiGLU/scales and native quantization metadata. The QAD NVFP4 source uses
FlashInfer packed weights, swizzled block scales and per-linear global scales;
it cannot be copied or renamed into native Comfy's fused format. In particular,
Q/K/V may have different global scales. Requantizing BF16 masters would be a
new PTQ treatment, not byte-preserving conversion of the released QAD kernel
representation. Neither unimplemented bridge should be advertised as a
working Comfy checkpoint.