TaoMate-R1

Reference-conditioned streaming audio-video generation built on MiniMax H3, developed by the Alibaba TaoLive AIGC Team.

The model generates audio and video jointly in three denoising steps per chunk. An opening LoRA handles the first chunk; a continuation LoRA handles later chunks with clean audio-video history. Both are online generator adapters, stored in float32 with LoRA rank 128 and alpha 128.

Files

config.json
cold/config.json
cold/adapter_model.safetensors
warm/config.json
warm/adapter_model.safetensors

Each bank contains 416 native H3 LoRA tensors. The runtime loads these adapters onto the MiniMax H3 Ref2VA base-model partition.

Usage

Install TaoMate-H3 using its shared installation instructions, then run the following commands from the repository root. Download the base partition and adapters:

hf download MiniMaxAI/MiniMax-H3 \
  --include "Ref2VA/*" --local-dir models/MiniMax-H3

hf download TaoLiveAIGC/TaoMate-R1 --local-dir models/TaoMate-R1

Run the included presenter-and-product example:

python -m taomate_h3.ref2av \
  --model-root models/MiniMax-H3 \
  --adapter models/TaoMate-R1 \
  --prompt-file examples/ref2av_prompt.txt \
  --reference-image examples/ref2av/presenter.png \
  --reference-image examples/ref2av/product.png \
  --duration 10 --resolution 768x1344 --seed 9111 \
  --gpus 8 --output outputs/ref2av

Repeat --reference-image or --reference-video for multiple visual references. Their command-line order is preserved. Reference-video audio is ignored; output audio is generated by the model. The result is outputs/ref2av/video.mp4.

Inputs and usage · Example video

License and acknowledgments

This release follows the MiniMax H3 Community License Agreement. We thank MiniMax, Qwen, PyTorch, and the Self Forcing authors for their models, implementations, and research.