lightx2v commited on
Commit
cd2d3b5
Β·
verified Β·
1 Parent(s): a8a33c5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -44
README.md CHANGED
@@ -45,12 +45,6 @@ The [official MiniMax-H3 repository](https://huggingface.co/MiniMaxAI/MiniMax-H3
45
  provides the upstream model weights. **LightX2V is the recommended inference entry
46
  point for the workflow documented here.**
47
 
48
- This is an unofficial, community-trained project and is not affiliated with or
49
- endorsed by MiniMax. It is intended as a local alternative for producing structured,
50
- H3-oriented T2VA prompts when the hosted H3-Context-IR service is unavailable or
51
- unsuitable for a workflow. It does not reproduce the proprietary H3-Context-IR system,
52
- and exact output parity is not guaranteed.
53
-
54
  ## What it does
55
 
56
  The adapter turns a short prompt plus the requested aspect ratio and duration into a
@@ -82,16 +76,8 @@ preserving the original intent.
82
  ## Status and roadmap
83
 
84
  - [x] T2VA prompt rewriting
85
- - [x] Duration conditioning from 4 to 15 seconds
86
- - [x] Aspect-ratio conditioning: `21:9`, `16:9`, `4:3`, `1:1`, `3:4`, `9:16`
87
- - [x] Single-prompt inference script
88
- - [x] End-to-end MiniMax-H3 generation through [LightX2V](https://github.com/ModelTC/LightX2V)
89
  - [ ] First/last-frame-to-audio-video (FL2VA) prompt rewriting
90
  - [ ] Reference-to-audio-video (Ref2VA) prompt rewriting
91
- - [ ] Image-to-audio-video and last-frame variants
92
- - [ ] Image, video, and audio reference inputs
93
- - [ ] Quantitative evaluation and a reproducible public benchmark suite
94
- - [ ] Smaller and quantized rewriter variants
95
 
96
  Only **T2VA** is supported by this release. The remaining MiniMax-H3 tasks require
97
  multimodal inputs and task-specific training data; they will be added in future work.
@@ -120,13 +106,6 @@ python infer.py \
120
  --greedy
121
  ```
122
 
123
- The default model IDs are:
124
-
125
- ```text
126
- Base model: Qwen/Qwen3.6-27B
127
- Adapter: lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA
128
- ```
129
-
130
  Local paths are supported as well:
131
 
132
  ```bash
@@ -157,19 +136,6 @@ For fair comparisons, use the same decoding mode and seed for every rewriter. Gr
157
  decoding is the simplest deterministic setting; sampling is enabled when `--greedy`
158
  is omitted.
159
 
160
- ## Output format
161
-
162
- The model is trained to return exactly three fields in this order:
163
-
164
- ```text
165
- integrated_multimodal_description: [Shot 1] ... [Shot 2] At 00:05.000, ...
166
- overall_soundscape: ...
167
- non_diegetic_music: ...
168
- ```
169
-
170
- Pass the entire three-field result to the MiniMax-H3 T2VA pipeline in
171
- [LightX2V](https://github.com/ModelTC/LightX2V).
172
-
173
  ## Generate audio-video with LightX2V
174
 
175
  ### 1. Install LightX2V
@@ -262,47 +228,42 @@ identical inference settings. Only the prompt-rewriting method should differ.
262
  ### Prompt 1
263
 
264
  > **Original prompt:** TODO β€” paste Prompt 1 here
265
- > **Conditions:** TODO β€” duration, aspect ratio, seed, and generation configuration
266
 
267
  | Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
268
  |---|---|---|---|
269
- | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt1/original.mp4"></video> | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt1/minimax_context_ir.mp4"></video> | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt1/qwen_base.mp4"></video> | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt1/ours.mp4"></video> |
270
 
271
  ### Prompt 2
272
 
273
  > **Original prompt:** TODO β€” paste Prompt 2 here
274
- > **Conditions:** TODO β€” duration, aspect ratio, seed, and generation configuration
275
 
276
  | Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
277
  |---|---|---|---|
278
- | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt2/original.mp4"></video> | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt2/minimax_context_ir.mp4"></video> | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt2/qwen_base.mp4"></video> | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt2/ours.mp4"></video> |
279
 
280
  ### Prompt 3
281
 
282
  > **Original prompt:** TODO β€” paste Prompt 3 here
283
- > **Conditions:** TODO β€” duration, aspect ratio, seed, and generation configuration
284
 
285
  | Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
286
  |---|---|---|---|
287
- | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt3/original.mp4"></video> | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt3/minimax_context_ir.mp4"></video> | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt3/qwen_base.mp4"></video> | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt3/ours.mp4"></video> |
288
 
289
  ### Prompt 4
290
 
291
  > **Original prompt:** TODO β€” paste Prompt 4 here
292
- > **Conditions:** TODO β€” duration, aspect ratio, seed, and generation configuration
293
 
294
  | Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
295
  |---|---|---|---|
296
- | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt4/original.mp4"></video> | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt4/minimax_context_ir.mp4"></video> | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt4/qwen_base.mp4"></video> | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt4/ours.mp4"></video> |
297
 
298
  ### Prompt 5
299
 
300
  > **Original prompt:** TODO β€” paste Prompt 5 here
301
- > **Conditions:** TODO β€” duration, aspect ratio, seed, and generation configuration
302
 
303
  | Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
304
  |---|---|---|---|
305
- | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt5/original.mp4"></video> | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt5/minimax_context_ir.mp4"></video> | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt5/qwen_base.mp4"></video> | <video controls src="https://huggingface.co/lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA/resolve/main/assets/examples/prompt5/ours.mp4"></video> |
306
 
307
  Expected video paths and filenames are documented in
308
  [`assets/examples/UPLOAD_LAYOUT.md`](assets/examples/UPLOAD_LAYOUT.md).
 
45
  provides the upstream model weights. **LightX2V is the recommended inference entry
46
  point for the workflow documented here.**
47
 
 
 
 
 
 
 
48
  ## What it does
49
 
50
  The adapter turns a short prompt plus the requested aspect ratio and duration into a
 
76
  ## Status and roadmap
77
 
78
  - [x] T2VA prompt rewriting
 
 
 
 
79
  - [ ] First/last-frame-to-audio-video (FL2VA) prompt rewriting
80
  - [ ] Reference-to-audio-video (Ref2VA) prompt rewriting
 
 
 
 
81
 
82
  Only **T2VA** is supported by this release. The remaining MiniMax-H3 tasks require
83
  multimodal inputs and task-specific training data; they will be added in future work.
 
106
  --greedy
107
  ```
108
 
 
 
 
 
 
 
 
109
  Local paths are supported as well:
110
 
111
  ```bash
 
136
  decoding is the simplest deterministic setting; sampling is enabled when `--greedy`
137
  is omitted.
138
 
 
 
 
 
 
 
 
 
 
 
 
 
 
139
  ## Generate audio-video with LightX2V
140
 
141
  ### 1. Install LightX2V
 
228
  ### Prompt 1
229
 
230
  > **Original prompt:** TODO β€” paste Prompt 1 here
 
231
 
232
  | Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
233
  |---|---|---|---|
234
+ | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/_gF8xSGc7-Z-3Ggsyk6w9.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/haF7Ay70aadZkKfq-at2i.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/Ztp7sdxWxMHLy3Sda6bjT.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/2pFOcxgj_WjvsnRMUcVWO.mp4"></video> |
235
 
236
  ### Prompt 2
237
 
238
  > **Original prompt:** TODO β€” paste Prompt 2 here
 
239
 
240
  | Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
241
  |---|---|---|---|
242
+ | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/9F0i0ZwRT8sST-weN0L_q.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/_QevAfxiAf8_-fuwRibKX.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/xmdrQXfSwNyORY6bkMAZ7.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/fEhON5N5rM6JBLNqcAs1G.mp4"></video> |
243
 
244
  ### Prompt 3
245
 
246
  > **Original prompt:** TODO β€” paste Prompt 3 here
 
247
 
248
  | Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
249
  |---|---|---|---|
250
+ | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/doirId736PnPloWRWu5o8.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/_N087UQHGD298jnBYSYoF.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/YVW8NrGF9H1-My6uTFWz9.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/VDjkgjZtfPeFiHYM0Vhks.mp4"></video> |
251
 
252
  ### Prompt 4
253
 
254
  > **Original prompt:** TODO β€” paste Prompt 4 here
 
255
 
256
  | Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
257
  |---|---|---|---|
258
+ | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/mOQ8QidVn7Fc64UYSBcZR.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/ULjXofNgjvs2aud70jdy8.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/JgjzFsmiMWEfI7P7hERaQ.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/Aze2ErBmM9MtsE7cRNAL5.mp4"></video> |
259
 
260
  ### Prompt 5
261
 
262
  > **Original prompt:** TODO β€” paste Prompt 5 here
 
263
 
264
  | Original Prompt (No Rewrite) | MiniMax Context-IR | Qwen3.6-27B Base Rewriter | **H3-T2VA Context Rewriter LoRA (Ours)** |
265
  |---|---|---|---|
266
+ | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/PfSo5D_lIe-ERk1yUS5GZ.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/iJkM5xdnfOojBo2gKQyEI.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/mFuOxCyycFw9ZwwiJQM6k.mp4"></video> | <video controls src="https://cdn-uploads.huggingface.co/production/uploads/680de13385293771bc57400b/xnDSTpEpofplVIRCI5eBY.mp4"></video> |
267
 
268
  Expected video paths and filenames are documented in
269
  [`assets/examples/UPLOAD_LAYOUT.md`](assets/examples/UPLOAD_LAYOUT.md).