mmh3testing / codemap.md
jhickman51's picture
Merge upstream MiniMax H3 updates and preserve beta5 checkpoint
b0eb306
|
Raw History Blame Contribute Delete
2.55 kB

A newer version of the Gradio SDK is available: 6.31.0

Upgrade

Repository Atlas: minimax-h3-loras

Purpose

This Hugging Face ZeroGPU Space runs the denoising half of MiniMax-H3. Each request calls multimodalart/qwen3vl-conditioner for prompt embeddings and token tags, then produces video and audio locally. The default denoiser is the pinned merged-Turbo bf16 10Eros-Max beta5 checkpoint; the original repository supplies VAEs and configuration.

Entry point and modules

  • app.py: imports spaces first, re-exports compatibility symbols, loads the model at startup, builds and launches the Gradio UI.
  • config.py: environment settings, checkpoint selection, canvas labels, NSFW LoRA catalog, and UI limits.
  • model.py: checkpoint and component loading, placement, AoTI decision, status, and decode hooks.
  • h3_checkpoint.py: exact beta5 tensor mapping, compressed timestep curve, and transformer construction.
  • pipeline.py: request flow, frame snapping, remote /encode, GPU generation, and video mux.
  • ui.py: Gradio controls and event wiring.
  • state.py, task.py, request.py: shared model state, per-request identity, and resource release.
  • lora/: LoRA fetch, conversion, registry, cache, validation, apply, and remove operations.
  • h3_split_blocks.py: Diffusers blocks for the split t2va and ref2va pipelines.
  • h3_aoti.py: optional AoTI for the original base checkpoint and separate build utilities.
  • tests/: CPU and source-level checks for checkpoint conversion, LoRA compatibility, and task isolation.

Request flow

  1. ui.build_ui() binds pipeline.generate(); the API preserves its positional arguments.
  2. pipeline._generate_impl() validates input and LoRA choices, snaps frames to 17*n+5, and calls the conditioner /encode.
  3. The conditioner returns a safetensors file with prompt_embeds, text_token_tags, and resolved dimensions in metadata.
  4. pipeline._generate() runs on ZeroGPU, applies selected LoRAs under the model mutation lock, denoises and decodes, then releases request resources.
  5. The CPU side muxes frames and audio into an MP4 and returns the download.

Cross-Space and checkpoint contracts

  • config.CANVASES labels and the safetensors wire format must match the conditioner.
  • H3_CHECKPOINT=10eros-turbo-beta5 uses seven sampling points and does not load a separate Turbo LoRA. 10eros-beta5 is non-Turbo; base uses the original transformer and optional Turbo LoRA.
  • The VAEs remain float32. The beta5 curve checkpoint skips the original AoTI package.
  • import spaces must precede CUDA initialization.