mpeg-diffusion / README.md
multimodalart's picture
multimodalart HF Staff
Upload README.md with huggingface_hub
e835f7f verified
|
Raw History Blame Contribute Delete
2.76 kB
metadata
title: MPEG-Diffusion
emoji: 🎞️
colorFrom: purple
colorTo: green
sdk: gradio
sdk_version: 6.32.0
app_file: app.py
python_version: '3.12'
startup_duration_timeout: 1h
short_description: Mixed-resolution text-to-video with a self-selected region
models:
  - johndpope/MiniMax-H3-MPEG-Diffusion

MPEG-Diffusion: Mixed-Resolution Video Generation

Text-to-video demo for MPEG-Diffusion (Menn et al.) β€” a model that generates video the way video codecs think about it:

  1. a low-resolution track denoises the whole frame cheaply,
  2. a tiny linear region head self-selects the region that deserves detail,
  3. a high-resolution track refines only that region through dual-resolution attention,
  4. the two tracks merge in pixel space β€” full-res detail where it matters, at roughly half the compute of rendering everything at high resolution.

This Space runs the paper's MiniMax-H3 port (Apache-2.0, vendored under lot/ + third_party/) with the published research weights from johndpope/MiniMax-H3-MPEG-Diffusion: the ridge-fitted KV up-maps and the v3 region head applied to the int8 ConvRot base DiT (Comfy-Org/MiniMax-H3), at the port's quality preset β€” 20 steps, the region chosen at step 16, 6 high-resolution steps, pixel-space merge, on a short reference-length clip. By default the model picks its own detail region; a manual box can be forced under Advanced.

Because the 62 GiB Qwen3-VL text encoder does not fit beside the transformer on one GPU, prompt conditioning runs on the qwen3vl-conditioner Space and this Space runs the sampler.

Credits

  • Paper: MPEG-Diffusion: Mixed-Resolution Video Generation β€” Menn et al., arXiv 2611.12069.
  • Code: the paper's MiniMax-H3 port, scrya-com/mpeg-diffusion (Apache-2.0).
  • Weights: johndpope/MiniMax-H3-MPEG-Diffusion β€” fitted up-maps + region head on the int8 ConvRot base.
  • Base model: MiniMax-H3, subject to the MiniMax H3 Community License β€” outputs are machine-generated and must be disclosed as such; the license does not permit use in the EU, UK, Republic of Korea or USA, and commercial use by companies above US$20M annual revenue needs written authorization from MiniMax.
  • Example prompts: portrait descriptions in the style of Pexels image alt-texts (Pexels content is free to use).