Blame for README.md

---
license: other
license_name: minimax-h3-community
license_link: LICENSE
tags:
- video-generation
- streaming
- interactive
- world-model
- minimax-h3
---

# StreamRender-H3: Interactive Coding Worlds with Real-time Streaming Diffusion Render

**Project page:** https://nvlabs.github.io/Sana/Sol-Engine/StreamRender-H3/

StreamRender-H3 splits an interactive world into a **Code Game Engine** and a **Streaming Diffusion Render**. The engine keeps the world state in code (maps, physics, rules) and streams a semantic map for every frame. A two-step streaming video model built on MiniMax-H3 Ref2VA renders those maps into photorealistic video as the player drives, with the look set by a reference image and a prompt.

- Two denoising steps per chunk in the distilled student.
- Diffusion render of 217.96 ms per streaming step on 8 GB200 GPUs, a 3.67× speedup over the PyTorch baseline.
- Fast Causal H3 VAE decodes 2 latent frames at 720p in 24 ms on one GB200, built by a self-improving agent loop.

## Released weights

This repository contains **two trained model components**:

| Directory | Model | Checkpoint |
| --- | --- | --- |
| `backbone/` | Streaming H3 renderer, full 50-layer EMA backbone with LoRA merged | 1008b / step224 |
| `causal_decoder/` | Fast Causal H3 VAE, a pruned and distilled 24-layer causal decoder | step6000 / 20261005-v1 |

The renderer uses two denoising evaluations with the S=2/W=2/C=2 streaming
policy. The decoder processes two available latent frames per call, using
cached past context without future latent frames. These are the trained
renderer and decoder checkpoints, not a bundle of unchanged upstream models.

## Download and use

Runtime code and deployment instructions:
[NVlabs/Sana — StreamRender-H3](https://github.com/NVlabs/Sana/tree/sol-engine/models/streamrender_h3).

```python
from huggingface_hub import snapshot_download
snapshot_download("Efficient-Large-Model/StreamRender-H3",
                  local_dir="./StreamRender-H3")
```

The download is approximately 72.75 GB. The renderer is stored as nine
safetensors shards, preserving all original tensor values. The current
runtime expects one merged backbone file. Reconstruct it with:

```bash
python StreamRender-H3/download_assets.py
```

Reconstruction needs another 66.28 GB of disk and verifies the original
SHA-256. Point the runtime manifest's `backbone` entry to
`backbone/1008b_step224_backbone_ema_merged.safetensors` and its
`causal_decoder` entry to `causal_decoder/causal_vit24_step6000.safetensors`.
Provide the remaining runtime dependencies separately as described in the
runtime README and `configs/assets.example.json`; this release contains
only the two trained model components. `SHA256SUMS` and `inventory.json`
provide integrity information for the released files.

## Upstream runtime dependencies

The trained-weight release does not include unchanged upstream models.
Download these components from their original sources:

| Runtime asset | Original download source |
| --- | --- |
| Qwen3-VL conditioning weights/config (`qwen_model`) | [MiniMax-H3 / Ref2VA/text_encoder](https://huggingface.co/MiniMaxAI/MiniMax-H3/tree/main/Ref2VA/text_encoder) — the H3-compatible conditioning checkpoint |
| Tokenizer (`tokenizer`) | [MiniMax-H3 / Ref2VA/tokenizer](https://huggingface.co/MiniMaxAI/MiniMax-H3/tree/main/Ref2VA/tokenizer) |
| Image/text processor (`qwen_processor`) | [MiniMax-H3 / Ref2VA/processor](https://huggingface.co/MiniMaxAI/MiniMax-H3/tree/main/Ref2VA/processor) |
| Native H3 video VAE (`native_vae`) | [MiniMax-H3 / Ref2VA/video_vae/source](https://huggingface.co/MiniMaxAI/MiniMax-H3/tree/main/Ref2VA/video_vae/source) — `model.safetensors`, used for decoder construction |
| Native H3 audio VAE (`audio_vae`) | [MiniMax-H3 / Ref2VA/audio_vae](https://huggingface.co/MiniMaxAI/MiniMax-H3/tree/main/Ref2VA/audio_vae) — `model.safetensors` |
| Official H3 TAE (`tae`) | [TAeHV / taeh3.pth](https://github.com/madebyollin/taehv/blob/62f7591f59dfbb4c3c02b7a621d180a9eeaba26c/taeh3.pth) — official GitHub release; no official HF model repository was verified |

The [Qwen3-VL upstream model](https://huggingface.co/Qwen/Qwen3-VL-32B-Instruct)
is the upstream architecture. For this runtime, use the H3-distributed
`Ref2VA/text_encoder` checkpoint above rather than assuming another Qwen
checkpoint is interchangeable.

`qwen_dcp` is a PyTorch distributed-checkpoint representation of those
conditioning weights, not a separate model. The current runtime loader
requires a compatible local DCP conversion; the public upstream HF download
provides safetensors, not the local DCP directory. Do not point
`qwen_dcp` at the safetensors directory.

Download the H3 dependencies without its original transformer weights:

```python
from huggingface_hub import snapshot_download
snapshot_download(
    "MiniMaxAI/MiniMax-H3", local_dir="./MiniMax-H3",
    allow_patterns=["Ref2VA/text_encoder/*", "Ref2VA/tokenizer/*",
                    "Ref2VA/processor/*", "Ref2VA/video_vae/*",
                    "Ref2VA/audio_vae/*"],
)
```

Download the verified official TAE weights:

```bash
curl -fL -o taeh3_official.pth \
  https://raw.githubusercontent.com/madebyollin/taehv/62f7591f59dfbb4c3c02b7a621d180a9eeaba26c/taeh3.pth
```

Fill the corresponding paths in the runtime asset manifest. These links
identify external dependencies; they do not add their weights to this release.

## License

These model weights are derivatives governed by the MiniMax H3 Community
License, including its territory, use and redistribution terms. See
`LICENSE` and `NOTICE`. The runtime code's license does not replace the
model license.


## Citation

```bibtex
@misc{streamrenderh3_2026,
  title        = {StreamRender-H3: Interactive Coding Worlds with Real-time Streaming Diffusion Render},
  author       = {Yanzuo Lu and Tian Ye and Yitong Li and Shuchen Xue and Haozhe Liu and Song Han and Enze Xie},
  year         = {2026},
  howpublished = {\url{https://nvlabs.github.io/Sana/Sol-Engine/StreamRender-H3/}},
  note         = {Project page}
}
```

## Acknowledgements

We thank the [MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3) team
for the base video generation model and native video codec. Our streaming
renderer and distilled causal decoder build on these components. Their
upstream license and attribution notices remain applicable.