license: other
license_name: minimax-h3-community
license_link: LICENSE
library_name: pytorch
pipeline_tag: image-to-video
base_model: MiniMaxAI/MiniMax-H3
tags:
- lora
- audio-video-generation
- reference-to-video
- streaming
- safetensors
TaoMate-R1
Reference-conditioned streaming audio-video generation built on MiniMax H3, developed by the Alibaba TaoLive AIGC Team.
The model generates audio and video jointly in three denoising steps per chunk. An opening LoRA handles the first chunk; a continuation LoRA handles later chunks with clean audio-video history. Both are online generator adapters, stored in float32 with LoRA rank 128 and alpha 128.
Files
config.json
cold/config.json
cold/adapter_model.safetensors
warm/config.json
warm/adapter_model.safetensors
Each bank contains 416 native H3 LoRA tensors. The runtime loads these adapters onto the MiniMax H3 Ref2VA base-model partition.
Usage
Install TaoMate-H3 using its shared installation instructions, then run the following commands from the repository root. Download the base partition and adapters:
hf download MiniMaxAI/MiniMax-H3 \
--include "Ref2VA/*" --local-dir models/MiniMax-H3
hf download TaoLiveAIGC/TaoMate-R1 --local-dir models/TaoMate-R1
Run the included presenter-and-product example:
python -m taomate_h3.ref2av \
--model-root models/MiniMax-H3 \
--adapter models/TaoMate-R1 \
--prompt-file examples/ref2av_prompt.txt \
--reference-image examples/ref2av/presenter.png \
--reference-image examples/ref2av/product.png \
--duration 10 --resolution 768x1344 --seed 9111 \
--gpus 8 --output outputs/ref2av
Repeat --reference-image or --reference-video for multiple visual references.
Their command-line order is preserved. Reference-video audio is ignored; output
audio is generated by the model. The result is outputs/ref2av/video.mp4.
Inputs and usage · Example video
License and acknowledgments
This release follows the MiniMax H3 Community License Agreement. We thank MiniMax, Qwen, PyTorch, and the Self Forcing authors for their models, implementations, and research.