invoker-render / README.md
multimodalart's picture
multimodalart HF Staff
Invoker-Render demo
10cdaf1 verified
|
Raw History Blame Contribute Delete
2.37 kB
metadata
title: Invoker-Render 1.0
emoji: 🎮
colorFrom: pink
colorTo: gray
sdk: gradio
sdk_version: 6.32.0
app_file: app.py
python_version: '3.12'
startup_duration_timeout: 1h
short_description: Render photoreal video from a structural proxy + first frame
models:
  - inclusionAI/invoker-render-1.0
  - MiniMaxAI/MiniMax-H3
  - depth-anything/Depth-Anything-V2-Small-hf
license: other

Invoker-Render 1.0

Demo of inclusionAI/invoker-render-1.0, the causal four-step streaming renderer of Code World Model 2.0 (paper, code). Inputs: a structural proxy video (24 FPS, 17·N+5 frames), an appearance anchor (the opening RGB frame, 1344×768) and a text description. Output: a 1344×768 rendered video.

How this Space runs it

A single-GPU port of the authors' native SGLang runtime (H3FixedHistorySession, recipe cwm_causal/v1): nine-latent bootstrap then five-latent chunks, four Euler steps (shift 12), a one-latent attention sink plus the twelve most recent latents of proxy + clean history in a KV cache, frame-aligned RoPE with the 47-latent reference shift. Components the model repo does not ship come from MiniMax-H3 (Ref2VA/): the native video VAE (BF16 encoder, posterior mean, native whole-video decode) and the Qwen3-VL condition encoder (layer-50 hidden state, the first image presented as <Picture 1>).

Deviations forced by one ZeroGPU card instead of the 8-GPU production profile:

  • the DiT's block linears and the Qwen3-VL decoder linears are FP8 weight-only (per-channel scales);
  • AdaLN modulations are precomputed in FP32 for the five timesteps the recipe uses;
  • the native whole-video VAE decode replaces the (unreleased) fast causal decoder.

The Real footage → depth proxy option turns any video into a relative-depth proxy with Depth Anything V2.

Credits & licenses

  • Invoker-Render weights and the vendored h3vae/ code (from CodeWorldModel's raven framework): Apache-2.0 and CC BY-NC 4.0 (non-commercial).
  • MiniMax-H3 VAE / Qwen3-VL encoder: MiniMax H3 Community License.
  • Example clips: CC0 footage from linoyts/repo-to-space-example-videos.