README.md

metadata

license: other
language:
  - en
  - ko
  - zh
  - ja
base_model:
  - Comfy-Org/MiniMax-H3
pipeline_tag: image-to-video
library_name: comfyui
tags:
  - minimax-h3
  - video-generation
  - audio-video-generation
  - image-to-video
  - diffusion-transformer
  - quantization
  - ternary
  - balanced-base3
  - rung-omega
  - theoremflow-omega
  - comfyui
  - multilingual

MiniMax-H3-Rung-Omega (5.976 BPW, 14.98 GiB Joint-tANS / 16.22 GiB Raw Base-243) + TheoremFlow-Ω (8.92x Speedup)

🌐 Language / 언어 / 语言 / 言語:   🇺🇸 English  |  🇰🇷 한국어  |  🇨🇳 中文 (简体)  |  🇯🇵 日本語

🏦 국내 계좌이체 후원 (Korean Bank Transfer): NH농협(농축협) 352-1556-0640-33 (예금주: 윤여원)  |  💛 카카오페이 QR 송금  |  ☕ Buy Me a Coffee

☕ Support Open Quantization & Inference Acceleration Research & Request Next Models:

• 🇺🇸 Want another model quantized with Rung Ω (4-Trit Base-243) + TheoremFlow-Ω acceleration? Leave your model request in a Buy Me a Coffee support note (buymeacoffee.com/tripleyoung) (Priority Queue ⚡) or open a thread in our Hugging Face Community Discussions!
• 🇰🇷 원하시는 다른 모델(비디오·오디오 DiT, LLM 등)의 Rung Ω 양자화 및 TheoremFlow-Ω 추론 가속이 필요하신가요? Buy Me a Coffee 후원 메시지(buymeacoffee.com/tripleyoung)에 모델명을 남겨주시면 **최우선 순위(Priority Queue ⚡)**로 양자화해 드립니다! (HF 커뮤니티 탭에서도 요청 및 소통 가능 · 한국어 전체 문서)
• 🇨🇳 希望我们将其他模型（视频/音频 DiT、LLM 等）进行 Rung Ω 量化与 TheoremFlow-Ω 加速吗？ 欢迎在 Buy Me a Coffee 赞助留言 (buymeacoffee.com/tripleyoung) 中写下您需要的模型名称（赞助请求享最优先队列 ⚡），或在 Hugging Face 社区讨论区 (Discussions) 留言交流！(查看中文完整文档)
• 🇯🇵 他のモデル（動画・音声 DiT、LLM など）の Rung Ω 量子化や TheoremFlow-Ω 高速化をご希望ですか？ Buy Me a Coffee のサポートメッセージ (buymeacoffee.com/tripleyoung) にご希望のモデル名をご記入いただければ、最優先キュー (Priority Queue ⚡) で量子化いたします！（HF コミュニティ Discussions でもリクエスト受付中・日本語ドキュメント全文）

yeowon/MiniMax-H3-Rung-Omega provides the 50-Block Pure Rung Ω Balanced Base-3 (4-Trit, Base-243 5-in-4-byte) quantized checkpoint and the TheoremFlow-Ω Mathematical Floor Inference Engine for the MiniMax-H3 FL2VA (First/Last-Frame-to-Video+Audio, 33.1B Total / 20.11B Active Parameters) multimodal Diffusion Transformer.

• Zero Fallback (200 / 200 Linear Layers in Pure TensorCoreRungOmegaTernaryLayout): Every single linear projection (attn.qkv_proj, attn.out_proj, mlp.fc1, mlp.fc2) across all 50 DiT blocks (blocks.0 .. blocks.49) is quantized in the Rung Ω 4-Trit Balanced Base-3 lattice ([-40, +40] with group_size=128 FP16 scales and convrot_groupsize=256), with 0 fallback NVFP4 layers.
• Smaller than Official #1 Recommended int8_convrot (19.53 GiB → 14.98 GiB Joint-tANS / 16.22 GiB Raw uint8): Achieves 2.50x compression vs. BF16 Teacher (37.46 GiB) and 1.30x compression vs. int8_convrot (19.53 GiB), while outperforming int8_convrot on Audio & Spatial-Temporal Top-1 Velocity Agreement.
• TheoremFlow-Ω (8.92x End-to-End Sampling Speedup): Accelerates 20-step joint Video (864×480, 124 frames @ 24fps) + Stereo Audio (32kHz) flow-matching generation from 625.4s → 70.11s while preserving 99.58% Video Latent Cosine Similarity (20.80 dB SNR) and crystal-clear synchronized speech/dialogue.

---

🎬 Demo Preview (Generated by Pure 50-Block Rung Ω + TheoremFlow-Ω in 70.11s)

🎬 Side-by-Side Audio & Video Comparison: Official int8_convrot (19.53 GiB, 434.1s) vs Ours Rung-Omega + TheoremFlow-Omega (14.96 GiB, 70.1s — Turn Sound ON 🔊)

• Full Video + Synchronized Korean Speech (.mp4): assets/MiniMax_H3_TheoremFlow_Omega_Champion.mp4
• Pure 50-Block 20-Step Baseline Video (.mp4): assets/MiniMax_H3_Rung_Omega_50blk_Champion.mp4
• Extracted 32kHz Stereo Audio Track (.wav): assets/MiniMax_H3_TheoremFlow_Omega_Champion_audio.wav

---

📊 Benchmark Results

1. Full 50-Block End-to-End Velocity Fidelity vs. Official #1 Recommended int8_convrot

Evaluated on held-out multimodal diffusion trajectories (Video + Audio + Text + Conditioning Keyframe across noise levels $\sigma \in [0.08, 0.92]$) against the unquantized BF16 Teacher (37.46 GiB):

Model / Format	Active Linear Layers	Effective BPW	Total Size (GiB)	Compression vs. BF16	Video Spatial Top-1 (↑)	Audio Patch Top-1 (↑)	Video Velocity Cosine (↑)	Audio Velocity Cosine (↑)	Video $R^2$ (↑)	Audio $R^2$ (↑)
BF16 Teacher (minimax_h3_fl2va_pruned_bf16)	200 × BF16	16.000	37.46 GiB	1.00x	100.00%	100.00%	1.000000	1.000000	100.00%	100.00%
Official #1 int8_convrot (19.53 GiB)	200 × INT8	8.003	19.53 GiB	1.92x	72.92%	77.60%	0.980832	0.985771	96.16%	97.10%
Rung Ω 4-Trit Pareto Champion (Ours)	200 × 4-Trit Base-243	5.967 (6.40 raw)	14.96 GiB (16.22 GiB raw)	2.50x	78.12% (+5.20%p)	81.77% (+4.17%p)	0.976383	0.982006	95.28%	96.43%

2. Multi-Track Pareto Ladder (10-Block Calibration & Held-Out Benchmark)

Track	Trits / Codebook	BPW (Joint-tANS)	Total Size (GiB)	Compression vs. BF16	Combined Top-1 (↑)	Combined SNR (dB)	Video Cosine (↑)	Audio Cosine (↑)
baseline_nvfp4_rtn	NVFP4	4.500	11.67 GiB	3.21x	97.53%	33.61 dB	0.999810	0.999676
omega_base15_ultra	3-Trit ([-7..+7], 2 w/byte)	3.432	9.27 GiB	4.04x	99.48%	44.31 dB	0.999984	0.999968
omega_4trit_sub45	4-Trit ECSQ ([-40..+40])	4.488	11.64 GiB	3.22x	99.48%	47.77 dB	0.999992	0.999990
omega_4trit_pareto (Released Checkpoint)	4-Trit Full-Range ([-40..+40])	5.977	14.98 GiB (16.22 GiB raw)	2.50x	99.87%	50.62 dB	0.999995	0.999996
omega_5trit_supreme	5-Trit ([-121..+121])	7.575	18.57 GiB	2.02x	99.48%	52.60 dB	0.999997	0.999998

3. TheoremFlow-Ω Inference Acceleration (864×480, 124 Frames + Stereo Audio, 20 Steps)

Engine / Schedule	Sampling Time (↓)	Speedup vs. Baseline	Video Latent Cosine	Video Latent SNR	Audio Latent Cosine
Standard 20-Step Full 50-Block Sampling	625.40s	1.00x	1.000000	∞	1.000000
TheoremFlow-Ω (6-Exact + 14-Fast, Floors 1–5)	70.11s	8.92x	0.995812	20.80 dB	0.971400

---

📦 Repository Structure

yeowon/MiniMax-H3-Rung-Omega/
├── diffusion_models/
│   └── minimax_h3_fl2va_pruned_rung_omega.safetensors   # 16.22 GiB (50-Block Pure Rung Ω 4-Trit Base-243 Checkpoint)
├── src/
│   ├── register_comfyui_rung_omega.py                   # ComfyUI TensorCoreRungOmegaTernaryLayout registration patch
│   ├── theoremflow_omega.py                             # TheoremFlow-Ω (Floors 1-5) 8.92x inference acceleration engine
│   └── generate_minimax_h3_rung_omega_video.py          # Standalone 3-stage Video+Audio generator with 5k+2 bounded VAE decode
├── benchmarks/
│   ├── rung_omega_minimax_h3_50blk_results.json         # Full 50-block evaluation metrics vs. int8_convrot
│   └── rung_omega_minimax_h3_results.json               # Multi-track Pareto ladder benchmark metrics
└── assets/
    ├── minimax_h3_rung_omega_benchmark_hero.png         # 50-Block Size, Velocity Top-1 & TheoremFlow-Ω 8.92x Speedup Chart
    ├── minimax_h3_pareto_frontier_chart.png             # Rate-Distortion Pareto Frontier vs. NVFP4 & int8_convrot Chart
    ├── MiniMax_H3_TheoremFlow_Omega_Champion.mp4        # 70.11s TheoremFlow-Ω generated video + Korean speech
    ├── MiniMax_H3_Rung_Omega_50blk_Champion.mp4         # Full 20-step generated video + Korean speech
    ├── MiniMax_H3_TheoremFlow_Omega_Champion_audio.wav  # 32kHz stereo audio output
    └── MiniMax_H3_TheoremFlow_Omega_Champion_keyframes.png

---

🚀 Quickstart

1. Register TensorCoreRungOmegaTernaryLayout in ComfyUI

Patch your ComfyUI installation (comfy/quant_ops.py and comfy/ops.py) so UNETLoader natively recognizes "format": "rung_omega_ternary":

python3 src/register_comfyui_rung_omega.py

2. Place Checkpoint in ComfyUI diffusion_models/

ln -s /path/to/diffusion_models/minimax_h3_fl2va_pruned_rung_omega.safetensors \
      /path/to/ComfyUI/models/diffusion_models/minimax_h3_fl2va_pruned_rung_omega.safetensors

3. Run End-to-End Video + Audio Generation with TheoremFlow-Ω (8.92x Speedup)

python3 src/generate_minimax_h3_rung_omega_video.py \
  --unet-name minimax_h3_fl2va_pruned_rung_omega.safetensors \
  --first-frame /path/to/input_frame.png \
  --width 864 --height 480 --length 124 --steps 20 \
  --theoremflow \
  --output-mp4 output_theoremflow_omega.mp4

4. Programmatic Usage of TheoremFlow-Ω on a Loaded ComfyUI Model

import torch
from theoremflow_omega import install_floor1_static_int8_engine, install_theoremflow_omega_schedule

# 1. Install Floor 1: Gauge-folded Base-243 -> Static-Workspace INT8 MMA + Fused SwiGLU + NHD SageAttention
f1_info = install_floor1_static_int8_engine(
    model.model.diffusion_model,
    max_seq_len=16384,
    device=torch.device("cuda:0"),
)

# 2. Install Floors 2, 3, and 5: Step-0 Constant Folding + Bi-Time Secant Boundary-Pinned Trunk Schedule
tf_state = install_theoremflow_omega_schedule(
    model.model.diffusion_model,
    exact_steps={0, 1, 2, 7, 13, 19},
    head_blocks=1,
    tail_blocks=1,
)

---

🔬 Technical Architecture Overview

1. Bijective 5-in-4 Base-243 Packing (rung_omega_ternary):
   • Each linear weight matrix $W \in \mathbb{R}^{M \times K}$ is rotated in 256-element blocks via the DC-balanced regular orthogonal Hadamard matrix $H_{256} = H_4^{\otimes 4}$ and mapped onto the symmetric 4-Trit Balanced Base-3 lattice $q \in {-40, \dots, +40}$ with FP16 group scales (group_size = 128).
   • Five 4-trit codes $(u_0, u_1, u_2, u_3, u_4) \in {0, \dots, 80}^5$ are bijectively packed into 4 bytes $b_i = u_i + 81 \cdot \text{trit}_i(u_4) \in {0, \dots, 242}$ (6.40 raw storage bits/weight, 5.976 Joint-tANS entropy bits/weight).
2. TheoremFlow-Ω Inference Engine (src/theoremflow_omega.py):
   • Floor 1 (Hardware Execution Roofline): Unpacks Base-243 weights once at load time into gauge-folded INT8 tensors (-3.55 GiB VRAM saved), pre-allocates a static contiguous CUDA workspace (eliminating 16,000+ cudaMalloc/cudaFree calls), fuses SwiGLU + ConvRot64 + Rowwise INT8 Quantize in MLP.fc2, and runs zero-transpose NHD sageattn_qk_int8_pv_fp16_cuda.
   • Floor 2 & 3 (Bi-Time Intrinsic Flow & Boundary-Pinned Trunk Splitting): Evaluates exact boundary blocks and FinalLayer on the live Step-$k$ state while bridging the interior trunk via modality-decoupled Bi-Time Secant extrapolation along $(t_v, t_a)$.
   • Floor 5 (Zero-Recompute Step-0 Constant Folding): Folds PackedLayout, rope_rotation_table, TokenRefiner(condition_proj(text_states)), and video_patch_proj(cond_video_rows) into Step-0 constants (0.0 ms overhead on Steps 1..19).

---

☕ Support Our Research & Request Custom Model Quantization

We release MiniMax-H3-Rung-Omega and TheoremFlow-Ω openly with zero gated access barriers so the community can run high-fidelity multimodal Video+Audio generation faster and in smaller VRAM footprints.

🎯 Want Another Model Quantized? (Rung Ω + TheoremFlow-Ω)

Have a large Video/Audio Diffusion Transformer (Wan, HunyuanVideo, LTX, Sora-class DiT) or LLM you want compressed to Rung Ω (3-Trit / 4-Trit / 5-Trit Base-243) with TheoremFlow-Ω acceleration?

1. ⚡ Priority Supporter Queue: Leave the target model name in your Buy Me a Coffee support note (buymeacoffee.com/tripleyoung) — supporter requests are prioritized first!
2. 💬 Community Channel & Model Requests: Open a discussion thread anytime in the Hugging Face Community Tab (Discussions) to request models, ask questions, or talk directly with us.

• ☕ Buy Me a Coffee (Global Support & Priority Requests): https://buymeacoffee.com/tripleyoung
• 💛 KakaoPay QR Transfer (한국 카카오페이 QR 송금 후원): https://qr.kakaopay.com/FVgCIrWbX
• 🏦 Korean Bank Transfer (국내 계좌이체 후원): NH농협(농축협) 352-1556-0640-33 (예금주: 윤여원)
• 💬 Community & Model Request Board: https://huggingface.co/yeowon/MiniMax-H3-Rung-Omega/discussions