show TE-only VRAM and showcase grids

cf43b8c verified 1 day ago

9.11 kB

language:
  - en
license: apache-2.0
library_name: comfyui
pipeline_tag: text-to-video
tags:
  - minimax-h3
  - text-encoder
  - qwen3-vl
  - int8
  - nvfp4
  - convrot
  - comfyui

Qwen3-VL Text Encoders for MiniMax-H3

Modified Qwen3-VL text encoders for MiniMax-H3 T2V in ComfyUI.

This repository contains only text encoders, recovery adapters, and evidence. It does not contain a MiniMax-H3 diffusion model or VAE.

Files

text_encoders/
├── recovered_8b/
│   ├── qwen3vl_8b_minimax_h3_recovered_bf16.safetensors
│   ├── qwen3vl_8b_minimax_h3_recovered_int8_convrot.safetensors
│   ├── ara.safetensors
│   ├── conditioning_adapter.safetensors
│   └── minimax_h3_recovered_8b_manifest.json
└── pruned24_balanced/
    ├── qwen3vl_32b_minimax_h3_pruned24_balanced_int8_convrot.safetensors
    ├── qwen3vl_32b_minimax_h3_pruned24_balanced_nvfp4_awq.safetensors
    ├── balanced_ara.safetensors
    └── minimax_h3_pruned24_balanced_manifest.json

Use in ComfyUI

Install kgonia/ComfyUI-MiniMaxH3TextEncoders. It adds one node:

Load MiniMax H3 Text Encoder

Select one encoder. Connect its CLIP output to the official MiniMaxH3ImageToVideo node.

All nodes after the loader are official ComfyUI MiniMax-H3 nodes.

The loader applies ARA and the recovered 8B 4096-to-5120 adapter internally. No adapter, ARA, cache, or barrier node is needed.

Showcase

Matched 10-second outputs, generated with the same prompt, seed, resolution, frame count, sampler, scheduler, and step count:

Official Qwen3-VL-32B NVFP4/AWQ	Recovered Qwen3-VL-8B BF16	Pruned-24 INT8 ConvRot
Video	Video	Video

30-second sequential comparison: official → recovered 8B → pruned-24

Twenty-frame grids

The original 6720-pixel-wide combined contact sheet is preserved at three_text_encoders_20_shots_each.png. The same 60 frames are split below into three readable 5×4 grids.

Official Qwen3-VL-32B NVFP4/AWQ

Recovered Qwen3-VL-8B BF16 + ARA + adapter

Pruned-24 INT8 ConvRot + balanced ARA

The showcase predates the pruned-24 NVFP4/AWQ release and does not provide a visual comparison for that storage variant. Numerical INT8-versus-NVFP4 conditioning parity is reported below.

Required official files

Download these from Comfy-Org/MiniMax-H3:

diffusion_models/minimax_h3_fl2va_pruned_int8_convrot.safetensors
vae/minimax_h3_video_vae_fp16.safetensors
vae/minimax_h3_audio_vae_fp32.safetensors

Encoder variants

Node choice	Base	Extra weights	Output width	TE peak VRAM
`recovered_8b: recovered_8b_bf16`	Qwen3-VL-8B BF16, 24 language layers	ARA + adapter	5120	`10,817 MiB` (`10.56 GiB`)
`recovered_8b: recovered_8b_int8_convrot`	Same recovered 8B architecture, INT8 ConvRot	ARA + adapter	5120	`6,333 MiB` (`6.18 GiB`)
`pruned24_balanced: int8_convrot`	Qwen3-VL-32B-derived sparse 24-block INT8 ConvRot	balanced ARA	5120	`15,525 MiB` (`15.16 GiB`)
`pruned24_balanced: nvfp4_awq`	Same retained blocks `0–8,35–49`, pruned directly from the official NVFP4/AWQ checkpoint	balanced ARA	5120	`10,150 MiB` (`9.91 GiB`)

ConvRot describes base-weight storage and runtime quantization. It does not replace the recovered 8B adapter. The adapter is a nonlinear 4096-to-5120 module and is executed by the loader.

VRAM validation

The old pruned-24 INT8 ConvRot artifact was physically pruned from the full 50-layer INT8 ConvRot checkpoint (bc2ced0…), not from the official NVFP4/AWQ checkpoint. Because each retained INT8 layer is larger than an NVFP4 layer, that artifact did not realize the expected VRAM reduction.

The NVFP4/AWQ variant is instead a sparse copy of the pinned official Comfy-Org/MiniMax-H3 checkpoint at revision 0543966fbdce5ba05709a8f2031c94bdba629b4a. It preserves all retained quantized tensors exactly and removes all tensors for original language blocks 9–34.

Clean external VRAM sampling used one fresh process per encoder, the same seven-token prompt, and 100 ms nvidia-smi sampling. Only the measured text-encoder contribution is reported:

Encoder	TE peak VRAM
Recovered 8B INT8 ConvRot	`6,333 MiB` (`6.18 GiB`)
Pruned-24 NVFP4/AWQ	`10,150 MiB` (`9.91 GiB`)
Recovered 8B BF16	`10,817 MiB` (`10.56 GiB`)
Pruned-24 INT8 ConvRot	`15,525 MiB` (`15.16 GiB`)
Official Qwen3-VL-32B NVFP4/AWQ reference	`16,430 MiB` (`16.04 GiB`)

The official reference is not a custom-node package choice; load it with the standard ComfyUI CLIPLoader. All four other rows are choices in Load MiniMax H3 Text Encoder. These measurements are short-prompt technical results, not general maximum requirements. Details: evidence/released_variant_metrics.json.

Validation and limits

The showcase used one matched 1344x768, 243-frame, 24 FPS, 25-step T2V contract. The three source outputs and merged comparison are technical assets. They do not establish a visual-quality winner. The recovered 8B showcase uses the BF16 base; the INT8 ConvRot variant has separate numerical conditioning-parity evidence.

Three-encoder teacher-conditioning cosine

The original matched three-encoder comparison reported these mean cosine similarities against the official Qwen3-VL-32B NVFP4/AWQ conditioning reference:

Encoder	Cosine similarity to official reference
Official Qwen3-VL-32B NVFP4/AWQ reference	`1.000000000000000` (identity baseline)
Recovered Qwen3-VL-8B + ARA + 4096-to-5120 adapter	`0.938678461010568`
Physical pruned-24 Qwen3-VL-32B-derived INT8 ConvRot + balanced ARA	`0.9404705529707683`

These are text-conditioning diagnostics from the original three-encoder comparison. They are not visual-quality, prompt-adherence, or video-equivalence scores. Details: evidence/three_encoder_teacher_cosine.json.

Teacher-reference cosine was directly measured for the recovered BF16 and pruned INT8 variants used in that original comparison. It was not directly measured for their quantized siblings, so the table does not invent those values.

Same-architecture quantization parity

For one fixed seven-token prompt, each pair produced conditioning tensors with shape [1, 7, 5120]:

Pair	Cosine similarity
Recovered 8B BF16 vs INT8 ConvRot	`0.9999650716781616`
Pruned-24 INT8 ConvRot vs NVFP4/AWQ	`0.9999949932098389`

These pairwise values measure quantization parity. They are not substitutes for teacher-reference cosine and are not visual-quality scores. Details: evidence/released_variant_metrics.json.

Recovered-pair absolute-error details

For one fixed seven-token prompt, the recovered 8B BF16 and INT8 ConvRot variants produced conditioning tensors with shape [1, 7, 5120]. Their additional absolute-error measurements were:

Metric	Result
Mean absolute difference	`0.015660330653190613`
Maximum absolute difference	`0.6452102661132812`

This is a separate numerical conditioning-parity diagnostic for one prompt. It is not a visual-quality score, a prompt-adherence score, or proof that the INT8 variant produces identical video results. Full details are in evidence/release_loader_smoke.json.

The requested hard cuts at 3 and 7 seconds did not pass for any encoder. Read evidence/cut_diagnostics.json before using the showcase as prompt-adherence evidence.

The recovered 8B and pruned-24 variants were validated for text-only T2V. Do not use them for image, first-frame, last-frame, or reference inputs.

License and provenance

The Qwen-derived text-encoder weights, ARA weights, and conditioning adapter in this repository are released under Apache-2.0. See LICENSE and NOTICE.

Recovered 8B source: Qwen/Qwen3-VL-8B-Instruct, revision 0c351dd01ed87e9c1b53cbc748cba10e6187ff3b.
Pruned-24 source: Qwen/Qwen3-VL-32B-Instruct, revision 0cfaf48183f594c314753d30a4c4974bc75f3ccb.

These are unofficial modified derivatives and are not produced or endorsed by the Qwen team. MiniMax-H3 is the compatible downstream model. This repository does not contain MiniMax-H3 diffusion-model or VAE weights.