license: other
license_name: minimax-h3-community
license_link: LICENSE
tags:
- video-generation
- streaming
- interactive
- world-model
- minimax-h3
StreamRender-H3: Interactive Coding Worlds with Real-time Streaming Diffusion Render
Project page: https://nvlabs.github.io/Sana/Sol-Engine/StreamRender-H3/
StreamRender-H3 splits an interactive world into a Code Game Engine and a Streaming Diffusion Render. The engine keeps the world state in code (maps, physics, rules) and streams a semantic map for every frame. A two-step streaming video model built on MiniMax-H3 Ref2VA renders those maps into photorealistic video as the player drives, with the look set by a reference image and a prompt.
- Two denoising steps per chunk in the distilled student.
- Diffusion render of 217.96 ms per streaming step on 8 GB200 GPUs, a 3.67× speedup over the PyTorch baseline.
- Fast Causal H3 VAE decodes 2 latent frames at 720p in 24 ms on one GB200, built by a self-improving agent loop.
Released weights
This repository contains two trained model components:
| Directory | Model | Checkpoint |
|---|---|---|
backbone/ |
Streaming H3 renderer, full 50-layer EMA backbone with LoRA merged | 1008b / step224 |
causal_decoder/ |
Fast Causal H3 VAE, a pruned and distilled 24-layer causal decoder | step6000 / 20261005-v1 |
The renderer uses two denoising evaluations with the S=2/W=2/C=2 streaming policy. The decoder processes two available latent frames per call, using cached past context without future latent frames. These are the trained renderer and decoder checkpoints, not a bundle of unchanged upstream models.
Download and use
Runtime code and deployment instructions: NVlabs/Sana — StreamRender-H3.
from huggingface_hub import snapshot_download
snapshot_download("Efficient-Large-Model/StreamRender-H3",
local_dir="./StreamRender-H3")
The download is approximately 72.75 GB. The renderer is stored as nine safetensors shards, preserving all original tensor values. The current runtime expects one merged backbone file. Reconstruct it with:
python StreamRender-H3/download_assets.py
Reconstruction needs another 66.28 GB of disk and verifies the original
SHA-256. Point the runtime manifest's backbone entry to
backbone/1008b_step224_backbone_ema_merged.safetensors and its
causal_decoder entry to causal_decoder/causal_vit24_step6000.safetensors.
Provide the remaining runtime dependencies separately as described in the
runtime README and configs/assets.example.json; this release contains
only the two trained model components. SHA256SUMS and inventory.json
provide integrity information for the released files.
Upstream runtime dependencies
The trained-weight release does not include unchanged upstream models. Download these components from their original sources:
| Runtime asset | Original download source |
|---|---|
Qwen3-VL conditioning weights/config (qwen_model) |
MiniMax-H3 / Ref2VA/text_encoder — the H3-compatible conditioning checkpoint |
Tokenizer (tokenizer) |
MiniMax-H3 / Ref2VA/tokenizer |
Image/text processor (qwen_processor) |
MiniMax-H3 / Ref2VA/processor |
Native H3 video VAE (native_vae) |
MiniMax-H3 / Ref2VA/video_vae/source — model.safetensors, used for decoder construction |
Native H3 audio VAE (audio_vae) |
MiniMax-H3 / Ref2VA/audio_vae — model.safetensors |
Official H3 TAE (tae) |
TAeHV / taeh3.pth — official GitHub release; no official HF model repository was verified |
The Qwen3-VL upstream model
is the upstream architecture. For this runtime, use the H3-distributed
Ref2VA/text_encoder checkpoint above rather than assuming another Qwen
checkpoint is interchangeable.
qwen_dcp is a PyTorch distributed-checkpoint representation of those
conditioning weights, not a separate model. The current runtime loader
requires a compatible local DCP conversion; the public upstream HF download
provides safetensors, not the local DCP directory. Do not point
qwen_dcp at the safetensors directory.
Download the H3 dependencies without its original transformer weights:
from huggingface_hub import snapshot_download
snapshot_download(
"MiniMaxAI/MiniMax-H3", local_dir="./MiniMax-H3",
allow_patterns=["Ref2VA/text_encoder/*", "Ref2VA/tokenizer/*",
"Ref2VA/processor/*", "Ref2VA/video_vae/*",
"Ref2VA/audio_vae/*"],
)
Download the verified official TAE weights:
curl -fL -o taeh3_official.pth \
https://raw.githubusercontent.com/madebyollin/taehv/62f7591f59dfbb4c3c02b7a621d180a9eeaba26c/taeh3.pth
Fill the corresponding paths in the runtime asset manifest. These links identify external dependencies; they do not add their weights to this release.
License
These model weights are derivatives governed by the MiniMax H3 Community
License, including its territory, use and redistribution terms. See
LICENSE and NOTICE. The runtime code's license does not replace the
model license.
Citation
@misc{streamrenderh3_2026,
title = {StreamRender-H3: Interactive Coding Worlds with Real-time Streaming Diffusion Render},
author = {Yanzuo Lu and Tian Ye and Yitong Li and Shuchen Xue and Haozhe Liu and Song Han and Enze Xie},
year = {2026},
howpublished = {\url{https://nvlabs.github.io/Sana/Sol-Engine/StreamRender-H3/}},
note = {Project page}
}
Acknowledgements
We thank the MiniMax-H3 team for the base video generation model and native video codec. Our streaming renderer and distilled causal decoder build on these components. Their upstream license and attribution notices remain applicable.