--- license: other license_name: minimax-h3-community-license license_link: LICENSE base_model: MiniMaxAI/MiniMax-H3 tags: - mpeg-diffusion - mixed-resolution - lora - video-generation - research --- # MiniMax H3 two-track mixed resolution (MPEG-Diffusion) > [!WARNING] > **Research weights, help wanted.** Working now, no DiT fine-tune: the fitted K/V maps ([`kv_maps/`](kv_maps)) and region head v3 ([`region_head_v3/`](region_head_v3)) give sharp subjects on 22-frame 1344×768 video at **1.94×** the speed of full resolution (RTX 5090). The LoRA is not published: its 500-step run changes the whole picture. Help and results: [scrya-com/mpeg-diffusion issue #2](https://github.com/scrya-com/mpeg-diffusion/issues/2). > [!CAUTION] > **License and territory.** Derived from MiniMax H3 under the [MiniMax H3 Community License Agreement](LICENSE) (see [`NOTICE`](NOTICE)). Use and redistribution are allowed **only outside the European Union, the United Kingdom, the Republic of Korea and the United States of America**. Commercial use above US$20M yearly revenue needs MiniMax's written authorization. Do not use these weights to impersonate anyone, and disclose machine-generated output. Weights for two-track mixed-resolution sampling on the MiniMax H3 DiT. The method is MPEG-Diffusion: Mixed-Resolution Video Generation by Dennis Menn, Tianpei Gu, Guande He, Aryan V S, Yanke Song, Nan Liu, Desai Xie, Xun Huang, Chenfeng Xu and Diana Marculescu (UT Austin and Roblox), [arXiv 2611.12069](https://arxiv.org/abs/2611.12069). This repository is an independent port to H3, not their release. How it works: the unmodified H3 renders the whole frame at half resolution. A region, such as the face, gets full-resolution tokens. Dual-resolution attention lets every query read keys and values at its own resolution. The paper fine-tunes the model so the full-resolution track can start earlier and redraw detail. These weights are that fine-tune, made affordable as a rank-16 LoRA plus the K/V up-maps and two resolution vectors. | Folder | What | |---|---| | [`kv_maps/`](kv_maps) | Fitted `f_up` K/V maps (rank 128, 50 blocks). They remove the diamond-grid grain of copied keys in the full-resolution region. Use with `render --preset quality` or the ComfyUI node's `maps` input. | | [`region_head_v3/`](region_head_v3) | Region head v3 (see its card); the one the quality recipe uses. | | [`region_head/`](region_head) | The self-selected region head: one linear layer on block 25 of the frozen DiT that marks the subject partway through sampling, and so sets the full-resolution region and the Level-of-Token layout. Trained on 2,354 pictures with YOLOE subject labels; held-out mIoU 0.468. It favours the centre of the frame (see its card). | | LoRA (not published) | Lets the full-resolution track start earlier. First run (2,346 pairs, RTX 5090) stopped at step 500 with held-out loss down 21% (half resolution) and 26% (full resolution). | - Code, sampler and training pipeline: [scrya-com/mpeg-diffusion](https://github.com/scrya-com/mpeg-diffusion) (`lot/two_track_h3.py`, `lot/train_tt.py`, one command: `lot/train_tt_all.sh`). - Precomputed caption embeddings for the Pexels photos, which skip the 32B text encoder: [johndpope/MiniMax-H3-pexels-people-text](https://huggingface.co/datasets/johndpope/MiniMax-H3-pexels-people-text). - Write-up: [scrya.com/mpeg_diffusion](https://scrya.com/mpeg_diffusion). - The earlier Level-of-Token adapters are in [johndpope/MiniMax-H3-LoT](https://huggingface.co/johndpope/MiniMax-H3-LoT).