SMD-MiniMax-H3-Ref2VA-4step

4-step SMD for image-guided video and audio generation. The student adapters, standalone inference code and reproducible examples are included. Upstream base components are downloaded separately.

Training code · SMD collection

Example

A woman and a Samoyed, supplied as separate references, share a playful moment on a snowy mountain overlook.

Reference subjects	Generated video · 4 steps

Person reference · Dog reference

The references define the subjects' appearance; the model generates their new poses, shared setting, interaction and sound. They are not first-frame anchors.

Download video with audio

Seed: 42. Size: 1344 × 768. Frames: 124 at 24 fps. Steps: 4.

Full prompt and case settings · Generation receipt

The separate reference images were generated with SMD-FLUX.2-Klein-4B, using the teacher-9b variant. Reference generation settings.

Models

Variant	Weights	Steps	Size
default	LoRA	4	1.38 GB

Run

For a native ComfyUI workflow on a single GPU, use the corresponding SMD-MiniMax-H3-Ref2VA-4step-ComfyUI. It includes a Compact profile and a BF16 transformer profile. The Python runner below uses two GPUs.

Base: MiniMaxAI/MiniMax-H3.

Use Linux, Python 3.12, Git and two NVIDIA GPUs with CUDA support. The requirements install Diffusers from a pinned Git revision. The included inference code does not import the SMD training repository.

python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install huggingface-hub
# Sign in to Hugging Face.
hf auth login
hf download zjwfufu/SMD-MiniMax-H3-Ref2VA-4step --local-dir SMD-MiniMax-H3-Ref2VA-4step --exclude "lora/adapter_model.safetensors"
cd SMD-MiniMax-H3-Ref2VA-4step
python -m pip install -r requirements.txt
python infer.py --output outputs

The runner requires two CUDA GPUs and launches two processes with FSDP2. It was tested on 2 × A100 80 GB, with 433 GiB of host RAM. It downloads only the selected transformer plus shared encoders, decoders and processor files to the Hugging Face cache. The selected base components occupy approximately 144 GB (134 GiB) per workflow; keeping both workflows reuses the shared components and requires approximately 210 GB (196 GiB) in total. Loading and fusing these weights can take longer than the sampling steps. --base selects a complete local upstream snapshot, and --case selects another case JSON.

The runner prefers smd_lora.safetensors at the repository root and falls back to the retained lora/adapter_model.safetensors path. Both files have identical bytes. The download command excludes the legacy duplicate; adapter fusion and sampling settings are unchanged.

Sampling

The released sampler uses raw final-segment scale 0.5. The raw grid is built before the model-specific time shift. The included runner follows sampling.json; changing only an upstream pipeline's step count does not reproduce the student.

Video shift is 12; audio shift is 3. LoRA scale is 1.0. The default case uses 124 frames at 24 fps and joint video/audio generation.

The supplied loader validates and applies all LoRA matrices and saved replacement tensors, with the recorded merge precision. Omitting replacement tensors or using a different base can change the result.

License and attribution

Base components retain the terms described in the MiniMaxAI/MiniMax-H3 model card. Original code notices and bundled licenses are retained. Adapter use remains subject to the applicable base-model terms.