-352--1556--0640--33%20(%EC%9C%A4%EC%97%AC%EC%9B%90)-0072BC?style=for-the-badge){kind=icon}
๐ฆ ๊ตญ๋ด ๊ณ์ข์ด์ฒด ํ์ (Korean Bank Transfer): NH๋ํ(๋์ถํ) 352-1556-0640-33 (์๊ธ์ฃผ: ์ค์ฌ์) |
๐ ์นด์นด์คํ์ด QR ์ก๊ธ |
โ Buy Me a Coffee
๐ Language / ์ธ์ด / ่ฏญ่จ / ่จ่ช: ๐บ๐ธ English | ๐ฐ๐ท ํ๊ตญ์ด | ๐จ๐ณ ไธญๆ (็ฎไฝ) | ๐ฏ๐ต ๆฅๆฌ่ช
๐ฆ ๊ตญ๋ด ๊ณ์ข์ด์ฒด ํ์ (Korean Bank Transfer): NH๋ํ(๋์ถํ) 352-1556-0640-33 (์๊ธ์ฃผ: ์ค์ฌ์) |
๐ ์นด์นด์คํ์ด QR ์ก๊ธ |
โ Buy Me a Coffee
* **Full Video + Synchronized Korean Speech (`.mp4`)**: [`assets/MiniMax_H3_TheoremFlow_Omega_Champion.mp4`](./assets/MiniMax_H3_TheoremFlow_Omega_Champion.mp4) * **Pure 50-Block 20-Step Baseline Video (`.mp4`)**: [`assets/MiniMax_H3_Rung_Omega_50blk_Champion.mp4`](./assets/MiniMax_H3_Rung_Omega_50blk_Champion.mp4) * **Extracted 32kHz Stereo Audio Track (`.wav`)**: [`assets/MiniMax_H3_TheoremFlow_Omega_Champion_audio.wav`](./assets/MiniMax_H3_TheoremFlow_Omega_Champion_audio.wav) --- ## ๐ Benchmark Results  ### 1. Full 50-Block End-to-End Velocity Fidelity vs. Official `#1` Recommended `int8_convrot` Evaluated on held-out multimodal diffusion trajectories (`Video + Audio + Text + Conditioning Keyframe` across noise levels $\sigma \in [0.08, 0.92]$) against the unquantized `BF16` Teacher (`37.46 GiB`): | Model / Format | Active Linear Layers | Effective BPW | Total Size (GiB) | Compression vs. BF16 | Video Spatial Top-1 (โ) | Audio Patch Top-1 (โ) | Video Velocity Cosine (โ) | Audio Velocity Cosine (โ) | Video $R^2$ (โ) | Audio $R^2$ (โ) | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | **BF16 Teacher (`minimax_h3_fl2va_pruned_bf16`)** | 200 ร BF16 | `16.000` | `37.46 GiB` | `1.00x` | `100.00%` | `100.00%` | `1.000000` | `1.000000` | `100.00%` | `100.00%` | | **Official `#1` `int8_convrot` (`19.53 GiB`)** | 200 ร INT8 | `8.003` | `19.53 GiB` | `1.92x` | `72.92%` | `77.60%` | `0.980832` | `0.985771` | `96.16%` | `97.10%` | | **`Rung ฮฉ` 4-Trit Pareto Champion (Ours)** | **200 ร 4-Trit Base-243** | **`5.967` (`6.40` raw)** | **`14.96 GiB` (`16.22 GiB` raw)** | **`2.50x`** | **`78.12%` (`+5.20%p`)** | **`81.77%` (`+4.17%p`)** | **`0.976383`** | **`0.982006`** | **`95.28%`** | **`96.43%`** | ### 2. Multi-Track Pareto Ladder (10-Block Calibration & Held-Out Benchmark)  | Track | Trits / Codebook | BPW (Joint-tANS) | Total Size (GiB) | Compression vs. BF16 | Combined Top-1 (โ) | Combined SNR (dB) | Video Cosine (โ) | Audio Cosine (โ) | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | | `baseline_nvfp4_rtn` | NVFP4 | `4.500` | `11.67 GiB` | `3.21x` | `97.53%` | `33.61 dB` | `0.999810` | `0.999676` | | **`omega_base15_ultra`** | 3-Trit (`[-7..+7]`, 2 w/byte) | **`3.432`** | **`9.27 GiB`** | **`4.04x`** | **`99.48%`** | **`44.31 dB`** | **`0.999984`** | **`0.999968`** | | **`omega_4trit_sub45`** | 4-Trit ECSQ (`[-40..+40]`) | **`4.488`** | **`11.64 GiB`** | **`3.22x`** | **`99.48%`** | **`47.77 dB`** | **`0.999992`** | **`0.999990`** | | **`omega_4trit_pareto` (Released Checkpoint)** | 4-Trit Full-Range (`[-40..+40]`) | **`5.977`** | **`14.98 GiB` (`16.22 GiB` raw)** | **`2.50x`** | **`99.87%`** | **`50.62 dB`** | **`0.999995`** | **`0.999996`** | | **`omega_5trit_supreme`** | 5-Trit (`[-121..+121]`) | **`7.575`** | **`18.57 GiB`** | **`2.02x`** | **`99.48%`** | **`52.60 dB`** | **`0.999997`** | **`0.999998`** | ### 3. `TheoremFlow-ฮฉ` Inference Acceleration (`864ร480`, `124` Frames + Stereo Audio, 20 Steps) | Engine / Schedule | Sampling Time (โ) | Speedup vs. Baseline | Video Latent Cosine | Video Latent SNR | Audio Latent Cosine | | :--- | :---: | :---: | :---: | :---: | :---: | | Standard 20-Step Full 50-Block Sampling | `625.40s` | `1.00x` | `1.000000` | `โ` | `1.000000` | | **`TheoremFlow-ฮฉ` (`6-Exact + 14-Fast`, Floors 1โ5)** | **`70.11s`** | **`8.92x`** | **`0.995812`** | **`20.80 dB`** | **`0.971400`** | --- ## ๐ฆ Repository Structure ```text yeowon/MiniMax-H3-Rung-Omega/ โโโ diffusion_models/ โ โโโ minimax_h3_fl2va_pruned_rung_omega.safetensors # 16.22 GiB (50-Block Pure Rung ฮฉ 4-Trit Base-243 Checkpoint) โโโ src/ โ โโโ register_comfyui_rung_omega.py # ComfyUI TensorCoreRungOmegaTernaryLayout registration patch โ โโโ theoremflow_omega.py # TheoremFlow-ฮฉ (Floors 1-5) 8.92x inference acceleration engine โ โโโ generate_minimax_h3_rung_omega_video.py # Standalone 3-stage Video+Audio generator with 5k+2 bounded VAE decode โโโ benchmarks/ โ โโโ rung_omega_minimax_h3_50blk_results.json # Full 50-block evaluation metrics vs. int8_convrot โ โโโ rung_omega_minimax_h3_results.json # Multi-track Pareto ladder benchmark metrics โโโ assets/ โโโ minimax_h3_rung_omega_benchmark_hero.png # 50-Block Size, Velocity Top-1 & TheoremFlow-ฮฉ 8.92x Speedup Chart โโโ minimax_h3_pareto_frontier_chart.png # Rate-Distortion Pareto Frontier vs. NVFP4 & int8_convrot Chart โโโ MiniMax_H3_TheoremFlow_Omega_Champion.mp4 # 70.11s TheoremFlow-ฮฉ generated video + Korean speech โโโ MiniMax_H3_Rung_Omega_50blk_Champion.mp4 # Full 20-step generated video + Korean speech โโโ MiniMax_H3_TheoremFlow_Omega_Champion_audio.wav # 32kHz stereo audio output โโโ MiniMax_H3_TheoremFlow_Omega_Champion_keyframes.png ``` --- ## ๐ Quickstart ### 1. Register `TensorCoreRungOmegaTernaryLayout` in ComfyUI Patch your ComfyUI installation (`comfy/quant_ops.py` and `comfy/ops.py`) so `UNETLoader` natively recognizes `"format": "rung_omega_ternary"`: ```bash python3 src/register_comfyui_rung_omega.py ``` ### 2. Place Checkpoint in ComfyUI `diffusion_models/` ```bash ln -s /path/to/diffusion_models/minimax_h3_fl2va_pruned_rung_omega.safetensors \ /path/to/ComfyUI/models/diffusion_models/minimax_h3_fl2va_pruned_rung_omega.safetensors ``` ### 3. Run End-to-End Video + Audio Generation with `TheoremFlow-ฮฉ` (`8.92x` Speedup) ```bash python3 src/generate_minimax_h3_rung_omega_video.py \ --unet-name minimax_h3_fl2va_pruned_rung_omega.safetensors \ --first-frame /path/to/input_frame.png \ --width 864 --height 480 --length 124 --steps 20 \ --theoremflow \ --output-mp4 output_theoremflow_omega.mp4 ``` ### 4. Programmatic Usage of `TheoremFlow-ฮฉ` on a Loaded ComfyUI Model ```python import torch from theoremflow_omega import install_floor1_static_int8_engine, install_theoremflow_omega_schedule # 1. Install Floor 1: Gauge-folded Base-243 -> Static-Workspace INT8 MMA + Fused SwiGLU + NHD SageAttention f1_info = install_floor1_static_int8_engine( model.model.diffusion_model, max_seq_len=16384, device=torch.device("cuda:0"), ) # 2. Install Floors 2, 3, and 5: Step-0 Constant Folding + Bi-Time Secant Boundary-Pinned Trunk Schedule tf_state = install_theoremflow_omega_schedule( model.model.diffusion_model, exact_steps={0, 1, 2, 7, 13, 19}, head_blocks=1, tail_blocks=1, ) ``` --- ## ๐ฌ Technical Architecture Overview 1. **Bijective 5-in-4 Base-243 Packing (`rung_omega_ternary`)**: * Each linear weight matrix $W \in \mathbb{R}^{M \times K}$ is rotated in 256-element blocks via the DC-balanced regular orthogonal Hadamard matrix $H_{256} = H_4^{\otimes 4}$ and mapped onto the symmetric 4-Trit Balanced Base-3 lattice $q \in \{-40, \dots, +40\}$ with `FP16` group scales (`group_size = 128`). * Five 4-trit codes $(u_0, u_1, u_2, u_3, u_4) \in \{0, \dots, 80\}^5$ are bijectively packed into 4 bytes $b_i = u_i + 81 \cdot \text{trit}_i(u_4) \in \{0, \dots, 242\}$ (`6.40` raw storage bits/weight, `5.976` Joint-tANS entropy bits/weight). 2. **TheoremFlow-ฮฉ Inference Engine (`src/theoremflow_omega.py`)**: * **Floor 1 (Hardware Execution Roofline)**: Unpacks Base-243 weights once at load time into gauge-folded INT8 tensors (`-3.55 GiB` VRAM saved), pre-allocates a static contiguous CUDA workspace (eliminating 16,000+ `cudaMalloc`/`cudaFree` calls), fuses `SwiGLU + ConvRot64 + Rowwise INT8 Quantize` in `MLP.fc2`, and runs zero-transpose `NHD` `sageattn_qk_int8_pv_fp16_cuda`. * **Floor 2 & 3 (Bi-Time Intrinsic Flow & Boundary-Pinned Trunk Splitting)**: Evaluates exact boundary blocks and `FinalLayer` on the live Step-$k$ state while bridging the interior trunk via modality-decoupled Bi-Time Secant extrapolation along $(t_v, t_a)$. * **Floor 5 (Zero-Recompute Step-0 Constant Folding)**: Folds `PackedLayout`, `rope_rotation_table`, `TokenRefiner(condition_proj(text_states))`, and `video_patch_proj(cond_video_rows)` into Step-0 constants (`0.0 ms` overhead on Steps `1..19`). --- ## โ Support Our Research & Request Custom Model Quantization We release **`MiniMax-H3-Rung-Omega`** and **`TheoremFlow-ฮฉ`** openly with zero gated access barriers so the community can run high-fidelity multimodal Video+Audio generation faster and in smaller VRAM footprints. ### ๐ฏ Want Another Model Quantized? (`Rung ฮฉ` + `TheoremFlow-ฮฉ`) Have a large Video/Audio Diffusion Transformer (Wan, HunyuanVideo, LTX, Sora-class DiT) or LLM you want compressed to **`Rung ฮฉ` (`3-Trit` / `4-Trit` / `5-Trit` Base-243)** with **`TheoremFlow-ฮฉ` acceleration**? 1. **โก Priority Supporter Queue**: Leave the target model name in your [**Buy Me a Coffee support note (`buymeacoffee.com/tripleyoung`)**](https://buymeacoffee.com/tripleyoung) โ supporter requests are prioritized first! 2. **๐ฌ Community Channel & Model Requests**: Open a discussion thread anytime in the [**Hugging Face Community Tab (`Discussions`)**](https://huggingface.co/yeowon/MiniMax-H3-Rung-Omega/discussions) to request models, ask questions, or talk directly with us.
