---
license: other
license_name: minimax-h3-community
license_link: LICENSE
library_name: pytorch
pipeline_tag: image-to-video
base_model: MiniMaxAI/MiniMax-H3
tags:
  - lora
  - audio-video-generation
  - reference-to-video
  - streaming
  - safetensors
---

# TaoMate-R1

Reference-conditioned streaming audio-video generation built on
[MiniMax H3](https://huggingface.co/MiniMaxAI/MiniMax-H3), developed by the
[Alibaba TaoLive AIGC Team](https://github.com/TaoLiveAIGC).

The model generates audio and video jointly in three denoising steps per chunk.
An opening LoRA handles the first chunk; a continuation LoRA handles later chunks
with clean audio-video history. Both are online generator adapters, stored in
float32 with LoRA rank 128 and alpha 128.

## Files

```text
config.json
cold/config.json
cold/adapter_model.safetensors
warm/config.json
warm/adapter_model.safetensors
```

Each bank contains 416 native H3 LoRA tensors. The runtime loads these adapters
onto the MiniMax H3 **Ref2VA** base-model partition.

## Usage

Install [TaoMate-H3](https://github.com/TaoLiveAIGC/TaoMate-H3) using its shared
installation instructions, then run the following commands from the repository
root. Download the base partition and adapters:

```bash
hf download MiniMaxAI/MiniMax-H3 \
  --include "Ref2VA/*" --local-dir models/MiniMax-H3

hf download TaoLiveAIGC/TaoMate-R1 --local-dir models/TaoMate-R1
```

Run the included presenter-and-product example:

```bash
python -m taomate_h3.ref2av \
  --model-root models/MiniMax-H3 \
  --adapter models/TaoMate-R1 \
  --prompt-file examples/ref2av_prompt.txt \
  --reference-image examples/ref2av/presenter.png \
  --reference-image examples/ref2av/product.png \
  --duration 10 --resolution 768x1344 --seed 9111 \
  --gpus 8 --output outputs/ref2av
```

Repeat `--reference-image` or `--reference-video` for multiple visual references.
Their command-line order is preserved. Reference-video audio is ignored; output
audio is generated by the model. The result is `outputs/ref2av/video.mp4`.

[Inputs and usage](https://github.com/TaoLiveAIGC/TaoMate-H3#-ref2av-taomate-r1)
· [Example video](https://github.com/user-attachments/assets/8a95695b-3e6a-410f-a82d-e74169b06173)

## License and acknowledgments

This release follows the [MiniMax H3 Community License Agreement](LICENSE).
We thank MiniMax, Qwen, PyTorch, and the Self Forcing authors for their models,
implementations, and research.