| --- |
| license: other |
| license_name: minimax-h3-community-license |
| license_link: https://huggingface.co/Enderfga/SolarRenderer/blob/main/LICENSE |
| base_model: MiniMaxAI/MiniMax-H3 |
| pipeline_tag: video-to-video |
| tags: |
| - video-generation |
| - video-to-video |
| - structure-conditioned |
| - lora |
| --- |
| |
| <p align="center"><img src="https://huggingface.co/Enderfga/SolarRenderer/resolve/main/assets/teaser.gif" width="100%"/></p> |
|
|
| <h1 align="center">SolarRenderer: Few-Step Generative Reference Rendering</h1> |
|
|
| <p align="center"> |
| 🌐 <a href="https://enderfga.cn/SolarRenderer">Project Page</a> • |
| 🎮 <a href="https://huggingface.co/spaces/Enderfga/SolarRenderer">Online Demo</a> • |
| 📄 <a href="https://arxiv.org/abs/2610.28630">Paper</a> • |
| 💻 <a href="https://github.com/Enderfga/SolarRenderer">Code</a> • |
| 🤗 <a href="https://huggingface.co/datasets/Enderfga/SolarRenderer-Data">Data</a> |
| </p> |
|
|
| **SolarRenderer** turns a structure video (grey box, semantic map, skeletons, depth, body mesh, or a sketch it never |
| saw in training) into a finished video, with an optional first frame and a prompt deciding the look. These are its |
| LoRAs on **MiniMax-H3** (Ref2VA transformer): a control LoRA and a four-step LoRA, each in a full-clip version and a |
| block-causal streaming version. Together they render a clip in **four steps**. |
|
|
| ## Method |
|
|
| <p align="center"><img src="https://huggingface.co/Enderfga/SolarRenderer/resolve/main/assets/method.jpg" width="100%"/></p> |
|
|
| ## Files |
|
|
| | File | What it is | |
| |---|---| |
| | `solarrenderer_control_lora.safetensors` | control LoRA, rank 384: the structure video enters H3's reference pathway, frame-aligned | |
| | `solarrenderer_4step_lora.safetensors` | four-step LoRA, rank 384: fine-tuned from [DMAD](https://huggingface.co/ZhengmingYu/DMAD) by structure-guided distribution matching | |
| | `solarrenderer_causal_control_lora.safetensors` | causal control LoRA, rank 384: the control LoRA trained further with block-causal attention. Each block of up to 5 latent frames sees the prompt, the first frame, the structure of the last 6 blocks and the 5 blocks rendered before it, never a later block | |
| | `solarrenderer_causal_4step_lora.safetensors` | causal four-step LoRA, rank 384: the four-step LoRA post-trained, with the causal control LoRA frozen, on its own block-by-block rollouts by distribution matching with structure guidance (w = 1.5) toward the full-clip control model | |
|
|
| ## Quick Start |
|
|
| ```bash |
| git clone https://github.com/Enderfga/SolarRenderer && cd SolarRenderer |
| pip install -e . && pip install flash-attn==2.7.4.post1 --no-build-isolation |
| python scripts/prepare_h3.py --dir ./MiniMax-H3 |
| hf download Enderfga/SolarRenderer --local-dir ./weights |
| python scripts/render.py --h3 ./MiniMax-H3 \ |
| --control weights/solarrenderer_control_lora.safetensors \ |
| --fewstep weights/solarrenderer_4step_lora.safetensors \ |
| --structure structure.mp4 --first first.png --prompt prompt.txt --out render.mp4 |
| ``` |
|
|
| Output: 1344×768 (or 768×1344), 24 fps, 158 frames; about 3.5 minutes of sampling on one GPU. The prompt follows one |
| six-part form: see the [prompt guide](https://github.com/Enderfga/SolarRenderer/blob/main/docs/prompts.md). |
|
|
| **Streaming.** The two causal LoRAs render the video block by block, four steps per block; each block can be shown as |
| soon as it is done. Same inputs, one flag: |
|
|
| ```bash |
| python scripts/render.py --h3 ./MiniMax-H3 --causal \ |
| --control weights/solarrenderer_causal_control_lora.safetensors \ |
| --fewstep weights/solarrenderer_causal_4step_lora.safetensors \ |
| --structure structure.mp4 --first first.png --prompt prompt.txt --out render.mp4 |
| ``` |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{fang2026solarrenderer, |
| title = {SolarRenderer: Few-Step Generative Reference Rendering}, |
| author = {Fang, Guian and Qian, Shengju and Lu, Dongyue and |
| Huang, Junchao and Jiang, Lutao and Wang, Xin and |
| Song, Yiren and Jiang, Li and Shou, Mike Zheng}, |
| year = {2026}, |
| eprint = {2610.28630}, |
| archivePrefix = {arXiv} |
| } |
| ``` |
|
|
| ## License |
|
|
| The weights are LoRA adapters on MiniMax-H3 and are distributed under the MiniMax H3 Community License Agreement (see |
| LICENSE and NOTICE), including its territorial restrictions (it excludes the European Union, the United Kingdom, the |
| Republic of Korea and the United States) and its Acceptable Use Policy. The four-step LoRAs descend from the |
| public DMAD four-step LoRA and are released under the same license. The code is released under the Apache License 2.0. |
|
|
