PEFT
Safetensors
MiniMax H3
English
lora
prompt-rewriting
text-to-audio-video
audio-video-generation
Instructions to use lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.6-27B") model = PeftModel.from_pretrained(base_model, "lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -45,12 +45,6 @@ The [official MiniMax-H3 repository](https://huggingface.co/MiniMaxAI/MiniMax-H3
|
|
| 45 |
provides the upstream model weights. **LightX2V is the recommended inference entry
|
| 46 |
point for the workflow documented here.**
|
| 47 |
|
| 48 |
-
This is an unofficial, community-trained project and is not affiliated with or
|
| 49 |
-
endorsed by MiniMax. It is intended as a local alternative for producing structured,
|
| 50 |
-
H3-oriented T2VA prompts when the hosted H3-Context-IR service is unavailable or
|
| 51 |
-
unsuitable for a workflow. It does not reproduce the proprietary H3-Context-IR system,
|
| 52 |
-
and exact output parity is not guaranteed.
|
| 53 |
-
|
| 54 |
## What it does
|
| 55 |
|
| 56 |
The adapter turns a short prompt plus the requested aspect ratio and duration into a
|
|
@@ -82,16 +76,8 @@ preserving the original intent.
|
|
| 82 |
## Status and roadmap
|
| 83 |
|
| 84 |
- [x] T2VA prompt rewriting
|
| 85 |
-
- [x] Duration conditioning from 4 to 15 seconds
|
| 86 |
-
- [x] Aspect-ratio conditioning: `21:9`, `16:9`, `4:3`, `1:1`, `3:4`, `9:16`
|
| 87 |
-
- [x] Single-prompt inference script
|
| 88 |
-
- [x] End-to-end MiniMax-H3 generation through [LightX2V](https://github.com/ModelTC/LightX2V)
|
| 89 |
- [ ] First/last-frame-to-audio-video (FL2VA) prompt rewriting
|
| 90 |
- [ ] Reference-to-audio-video (Ref2VA) prompt rewriting
|
| 91 |
-
- [ ] Image-to-audio-video and last-frame variants
|
| 92 |
-
- [ ] Image, video, and audio reference inputs
|
| 93 |
-
- [ ] Quantitative evaluation and a reproducible public benchmark suite
|
| 94 |
-
- [ ] Smaller and quantized rewriter variants
|
| 95 |
|
| 96 |
Only **T2VA** is supported by this release. The remaining MiniMax-H3 tasks require
|
| 97 |
multimodal inputs and task-specific training data; they will be added in future work.
|
|
@@ -120,13 +106,6 @@ python infer.py \
|
|
| 120 |
--greedy
|
| 121 |
```
|
| 122 |
|
| 123 |
-
The default model IDs are:
|
| 124 |
-
|
| 125 |
-
```text
|
| 126 |
-
Base model: Qwen/Qwen3.6-27B
|
| 127 |
-
Adapter: lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA
|
| 128 |
-
```
|
| 129 |
-
|
| 130 |
Local paths are supported as well:
|
| 131 |
|
| 132 |
```bash
|
|
@@ -157,19 +136,6 @@ For fair comparisons, use the same decoding mode and seed for every rewriter. Gr
|
|
| 157 |
decoding is the simplest deterministic setting; sampling is enabled when `--greedy`
|
| 158 |
is omitted.
|
| 159 |
|
| 160 |
-
## Output format
|
| 161 |
-
|
| 162 |
-
The model is trained to return exactly three fields in this order:
|
| 163 |
-
|
| 164 |
-
```text
|
| 165 |
-
integrated_multimodal_description: [Shot 1] ... [Shot 2] At 00:05.000, ...
|
| 166 |
-
overall_soundscape: ...
|
| 167 |
-
non_diegetic_music: ...
|
| 168 |
-
```
|
| 169 |
-
|
| 170 |
-
Pass the entire three-field result to the MiniMax-H3 T2VA pipeline in
|
| 171 |
-
[LightX2V](https://github.com/ModelTC/LightX2V).
|
| 172 |
-
|
| 173 |
## Generate audio-video with LightX2V
|
| 174 |
|
| 175 |
### 1. Install LightX2V
|
|
@@ -262,47 +228,42 @@ identical inference settings. Only the prompt-rewriting method should differ.
|
|
| 262 |
### Prompt 1
|
| 263 |
|
| 264 |
> **Original prompt:** TODO β paste Prompt 1 here
|
| 265 |
-
> **Conditions:** TODO β duration, aspect ratio, seed, and generation configuration
|
| 266 |
|
| 267 |
| Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
|
| 268 |
|---|---|---|---|
|
| 269 |
-
| <video controls src="https://huggingface.co/
|
| 270 |
|
| 271 |
### Prompt 2
|
| 272 |
|
| 273 |
> **Original prompt:** TODO β paste Prompt 2 here
|
| 274 |
-
> **Conditions:** TODO β duration, aspect ratio, seed, and generation configuration
|
| 275 |
|
| 276 |
| Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
|
| 277 |
|---|---|---|---|
|
| 278 |
-
| <video controls src="https://huggingface.co/
|
| 279 |
|
| 280 |
### Prompt 3
|
| 281 |
|
| 282 |
> **Original prompt:** TODO β paste Prompt 3 here
|
| 283 |
-
> **Conditions:** TODO β duration, aspect ratio, seed, and generation configuration
|
| 284 |
|
| 285 |
| Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
|
| 286 |
|---|---|---|---|
|
| 287 |
-
| <video controls src="https://huggingface.co/
|
| 288 |
|
| 289 |
### Prompt 4
|
| 290 |
|
| 291 |
> **Original prompt:** TODO β paste Prompt 4 here
|
| 292 |
-
> **Conditions:** TODO β duration, aspect ratio, seed, and generation configuration
|
| 293 |
|
| 294 |
| Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
|
| 295 |
|---|---|---|---|
|
| 296 |
-
| <video controls src="https://huggingface.co/
|
| 297 |
|
| 298 |
### Prompt 5
|
| 299 |
|
| 300 |
> **Original prompt:** TODO β paste Prompt 5 here
|
| 301 |
-
> **Conditions:** TODO β duration, aspect ratio, seed, and generation configuration
|
| 302 |
|
| 303 |
| Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
|
| 304 |
|---|---|---|---|
|
| 305 |
-
| <video controls src="https://huggingface.co/
|
| 306 |
|
| 307 |
Expected video paths and filenames are documented in
|
| 308 |
[`assets/examples/UPLOAD_LAYOUT.md`](assets/examples/UPLOAD_LAYOUT.md).
|
|
|
|
| 45 |
provides the upstream model weights. **LightX2V is the recommended inference entry
|
| 46 |
point for the workflow documented here.**
|
| 47 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
## What it does
|
| 49 |
|
| 50 |
The adapter turns a short prompt plus the requested aspect ratio and duration into a
|
|
|
|
| 76 |
## Status and roadmap
|
| 77 |
|
| 78 |
- [x] T2VA prompt rewriting
|
|
|
|
|
|
|
|
|
|
|
|
|
| 79 |
- [ ] First/last-frame-to-audio-video (FL2VA) prompt rewriting
|
| 80 |
- [ ] Reference-to-audio-video (Ref2VA) prompt rewriting
|
|
|
|
|
|
|
|
|
|
|
|
|
| 81 |
|
| 82 |
Only **T2VA** is supported by this release. The remaining MiniMax-H3 tasks require
|
| 83 |
multimodal inputs and task-specific training data; they will be added in future work.
|
|
|
|
| 106 |
--greedy
|
| 107 |
```
|
| 108 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 109 |
Local paths are supported as well:
|
| 110 |
|
| 111 |
```bash
|
|
|
|
| 136 |
decoding is the simplest deterministic setting; sampling is enabled when `--greedy`
|
| 137 |
is omitted.
|
| 138 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 139 |
## Generate audio-video with LightX2V
|
| 140 |
|
| 141 |
### 1. Install LightX2V
|
|
|
|
| 228 |
### Prompt 1
|
| 229 |
|
| 230 |
> **Original prompt:** TODO β paste Prompt 1 here
|
|
|
|
| 231 |
|
| 232 |
| Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
|
| 233 |
|---|---|---|---|
|
| 234 |
+
| <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/_gF8xSGc7-Z-3Ggsyk6w9.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/haF7Ay70aadZkKfq-at2i.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/Ztp7sdxWxMHLy3Sda6bjT.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/2pFOcxgj_WjvsnRMUcVWO.mp4"></video> |
|
| 235 |
|
| 236 |
### Prompt 2
|
| 237 |
|
| 238 |
> **Original prompt:** TODO β paste Prompt 2 here
|
|
|
|
| 239 |
|
| 240 |
| Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
|
| 241 |
|---|---|---|---|
|
| 242 |
+
| <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/9F0i0ZwRT8sST-weN0L_q.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/_QevAfxiAf8_-fuwRibKX.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/xmdrQXfSwNyORY6bkMAZ7.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/fEhON5N5rM6JBLNqcAs1G.mp4"></video> |
|
| 243 |
|
| 244 |
### Prompt 3
|
| 245 |
|
| 246 |
> **Original prompt:** TODO β paste Prompt 3 here
|
|
|
|
| 247 |
|
| 248 |
| Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
|
| 249 |
|---|---|---|---|
|
| 250 |
+
| <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/doirId736PnPloWRWu5o8.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/_N087UQHGD298jnBYSYoF.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/YVW8NrGF9H1-My6uTFWz9.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/VDjkgjZtfPeFiHYM0Vhks.mp4"></video> |
|
| 251 |
|
| 252 |
### Prompt 4
|
| 253 |
|
| 254 |
> **Original prompt:** TODO β paste Prompt 4 here
|
|
|
|
| 255 |
|
| 256 |
| Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
|
| 257 |
|---|---|---|---|
|
| 258 |
+
| <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/mOQ8QidVn7Fc64UYSBcZR.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/ULjXofNgjvs2aud70jdy8.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/JgjzFsmiMWEfI7P7hERaQ.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/Aze2ErBmM9MtsE7cRNAL5.mp4"></video> |
|
| 259 |
|
| 260 |
### Prompt 5
|
| 261 |
|
| 262 |
> **Original prompt:** TODO β paste Prompt 5 here
|
|
|
|
| 263 |
|
| 264 |
| Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
|
| 265 |
|---|---|---|---|
|
| 266 |
+
| <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/PfSo5D_lIe-ERk1yUS5GZ.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/iJkM5xdnfOojBo2gKQyEI.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/mFuOxCyycFw9ZwwiJQM6k.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/xnDSTpEpofplVIRCI5eBY.mp4"></video> |
|
| 267 |
|
| 268 |
Expected video paths and filenames are documented in
|
| 269 |
[`assets/examples/UPLOAD_LAYOUT.md`](assets/examples/UPLOAD_LAYOUT.md).
|