VideoNeuMat β€” weights

Model weights for VideoNeuMat: Neural Material Extraction from Generative Video Models (SIGGRAPH 2026).

The pipeline turns a text prompt (or image) into an 81-frame 1024Γ—1024 material video with a fine-tuned Wan-2.1 video model, then a feed-forward LRM extracts a re-renderable neural material (BRDF + displacement) from that video.

VideoNeuMat teaser: generated materials and their reconstructed, re-renderable appearances

Watch the overview video

Files

File Description
t2v/videoneumat-t2v.safetensors Text-to-video material generator (27 GB)
t2v_lora/videoneumat-t2v-lora.safetensors Text-to-video LoRA material generator (293 MB)
i2v/videoneumat-i2v.safetensors Image-to-video material generator (31 GB)
lrm/videoneumat-lrm.pth LRM encoder β€” material video β†’ latent
lrm/videoneumat-decoder.pth Shared neural-material MLP decoder

All three generators produce the same 81-pose sparse-rig material video that the LRM consumes. For setup and downloads, see the code repository and its release/download_weights.sh script.

License

This repository uses different licenses for different components. The licenses listed in the model-card metadata apply to the components below; they are not alternative licenses for every file.

The LRM checkpoints, including the shared MLP decoder, are licensed under the Creative Commons Attribution NonCommercial 4.0 International (CC BY-NC 4.0) license. The Wan-based generator checkpoints retain Apache-2.0, the same license as the upstream Wan2.1 models.

Files License
lrm/videoneumat-lrm.pth, lrm/videoneumat-decoder.pth CC BY-NC 4.0
t2v/videoneumat-t2v.safetensors Apache-2.0
t2v_lora/videoneumat-t2v-lora.safetensors Apache-2.0
i2v/videoneumat-i2v.safetensors Apache-2.0

The CC BY-NC 4.0 license does not apply to the Wan-based generator checkpoints. Third-party components retain their original licenses.

The original VideoNeuMat code is also licensed under CC BY-NC 4.0, with third-party components retaining their original licenses. See LICENSE for this model repository's license scope and the full CC BY-NC 4.0 terms.

Citation

@inproceedings{xue2026videoneumat,
  author    = {Xue, Bowen and Hadadan, Saeed and Zeng, Zheng and Rousselle, Fabrice and Montazeri, Zahra and Hasan, Milos},
  title     = {VideoNeuMat: Neural Material Extraction from Generative Video Models},
  booktitle = {ACM SIGGRAPH 2026 Conference Papers},
  year      = {2026},
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support