intel-ai/Wan2.2-Animate-14B-Diffusers-MXFP4-AutoRound

Wan2.2-Animate-14B · MXFP4 (AutoRound RTN)

This repository contains an MXFP4 W4A4 quantized version of Wan-AI/Wan2.2-Animate-14B-Diffusers, produced with AutoRound and intended for Diffusers inference on NVIDIA Blackwell (sm_120).

Model Summary

Item	Value
Base model	Wan-AI/Wan2.2-Animate-14B-Diffusers
Task	Character animation / replacement (WanAnimatePipeline)
Quantization	AutoRound 0.17.0
Scheme	MXFP4, W4A4, group size 32
Calibration	None (pure RTN, --iters 0 --disable_opt_rtn)
Model size	22 GB (transformer 9 GB)
Inference	Diffusers 0.40 + AutoRound quantizer
Tested on	NVIDIA RTX PRO 5000 72GB (sm_120)

The text encoder, image encoder, VAE, tokenizer, and scheduler are unchanged.

Quantization Details

transformer/quantization_config.json:

{
  "bits": 4,
  "act_bits": 4,
  "data_type": "mx_fp",
  "act_data_type": "mx_fp",
  "group_size": 32,
  "act_group_size": 32,
  "sym": true,
  "act_sym": true,
  "act_dynamic": true,
  "block_name_to_quantize": "motion_encoder.res_blocks,motion_encoder.motion_network,blocks,face_adapter",
  "quant_method": "auto-round",
  "packing_format": "auto_round:llm_compressor"
}

Coverage:

• Quantized: 512 linear layers — 40 transformer blocks × 12 linears (attn1.to_q/k/v/out, attn2.to_q/k/v/out/add_k_proj/add_v_proj, ffn.net.0.proj, ffn.net.2) plus 8 face_adapter blocks × 4 linears. Weights are stored as FP4 E2M1 with per-32 E8M0 scales; activations are dynamically quantized to MXFP4 at runtime.
• Kept at higher precision: motion_encoder convolutions, time_embedder, scale_shift_table, norm1/2/3, rope, patch_embedding, condition_embedder, and the embedding/norm/conv layers of the face encoder.
• vae, text_encoder, image_encoder, tokenizer, scheduler are not quantized.

Quantization Command

export MODEL=Wan-AI/Wan2.2-Animate-14B-Diffusers
export OUTPUT_DIR=./Wan2.2-Animate-14B-Diffusers-MXFP4

CUDA_VISIBLE_DEVICES=0,1 auto_round \
  --model "$MODEL" \
  --scheme MXFP4 \
  --iters 0 --disable_opt_rtn --disable_model_free \
  --device_map 0,1 --low_gpu_mem_usage --batch_size 1 \
  --output_dir "$OUTPUT_DIR"

# auto-round 0.17.0 writes model-*.safetensors for this pipeline; Diffusers
# expects diffusion_pytorch_model*.safetensors.
python - <<'PY'
from auto_round.utils.model import rename_weights_files
rename_weights_files("./Wan2.2-Animate-14B-Diffusers-MXFP4/transformer")
PY

--disable_opt_rtn forces pure RTN (need_calib=False); without it, --iters 0 would default to OptRTN and require calibration data. No calibration is used here because WanAnimate needs image + pose_video + face_video, while the available opens2v set only provides image+prompt. Quantization used 2× RTX PRO 5000 72GB and takes about 1-2 minutes for this model. MODEL can be a local Diffusers directory instead of the HF ID.

Inference

Requirements:

pip install "diffusers==0.40.0" "auto-round>=0.17.0" transformers accelerate safetensors "imageio[ffmpeg]"

Minimal example (run from this repository root):

import torch
from diffusers import AutoencoderKLWan, WanAnimatePipeline
from diffusers.utils import export_to_video, load_image, load_video

MODEL_ID = "."
vae = AutoencoderKLWan.from_pretrained(MODEL_ID, subfolder="vae", torch_dtype=torch.float32)
pipe = WanAnimatePipeline.from_pretrained(
    MODEL_ID, torch_dtype=torch.bfloat16, vae=vae, device_map="balanced"
)
pipe.reset_device_map()
pipe.enable_model_cpu_offload(device="cuda")

image = load_image("examples/animate/src_ref.png")
pose_video = load_video("examples/animate/src_pose.mp4")
face_video = load_video("examples/animate/src_face.mp4")

frames = pipe(
    image=image,
    pose_video=pose_video,
    face_video=face_video,
    prompt="People in the video are doing actions.",
    height=720,
    width=1280,
    segment_frame_length=77,
    prev_segment_conditioning_frames=1,
    num_inference_steps=20,
    mode="animate",
    guidance_scale=1.0,
    generator=torch.Generator(device="cuda").manual_seed(42),
    output_type="np",
).frames[0]
export_to_video(frames, "animate.mp4", fps=30)

device_map="balanced" is required so the AutoRound quantizer selects the CUDA kernel. enable_model_cpu_offload(device="cuda") keeps peak VRAM around 60 GB at 720p; omit it if all components fit. Set PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True if you hit fragmentation OOMs.

Running the Official Example

The repository ships the preprocessed official example inputs under examples/animate/ and a ready-to-run script. The inputs were produced with the official wan/modules/animate/preprocess/preprocess_data.py (--resolution_area 1280 720).

python examples/run_example.py --backend native
python examples/run_example.py --backend sage_v1
python examples/run_example.py --backend sage_v2
python examples/run_example.py --backend sage_v3

--limit-frames, --steps, and --segment-frame-length are available for shorter smoke runs. For a true short smoke, set both --limit-frames 5 and --segment-frame-length 5; otherwise the 5-frame input is padded to a 77-frame segment.

Results

Official example: 1280×720, 106 frames, 20 steps, guidance_scale=1.0, seed 42, segment_frame_length=77. One RTX PRO 5000 72GB, CPU offload, wall time includes pipeline load and VAE encode/decode.

Attention backend	Wall time	MAE vs BF16	SSIM vs BF16
native (SDPA)	59.4 min	7.09	0.8650
SageAttention v1	57.3 min	6.93	0.8660
SageAttention v2	52.5 min	6.72	0.8688
SageAttention v3	52.9 min	22.22	0.7307

Validation: the native and sage_v3 commands above were re-run end-to-end on an RTX PRO 5000 72GB (sm_120) and are bit-identical to the original campaign run (MAE 0, SSIM 1.0).

SageAttention v3 uses Blackwell microscaling FP4 attention. It is the fastest backend but has visibly lower fidelity, including a brightness/lighting shift in the later frames. Use v1/v2 or native when quality matters.

Frames 0 / 40 / 80 / 105, MAE is against BF16 native. The v3 lighting shift becomes obvious in the last two rows.

Sample Outputs

Backend	Video
native	examples/mxfp4_native.mp4
SageAttention v1	examples/mxfp4_sage_v1.mp4
SageAttention v2	examples/mxfp4_sage_v2.mp4
SageAttention v3	examples/mxfp4_sage_v3.mp4

Limitations

• Pure RTN, no calibration: WanAnimate needs image + pose_video + face_video, and the available I2V calibration set (opens2v) only provides image+prompt. An OptRTN version would need a pose/face calibration set.
• All numbers were measured on one RTX PRO 5000 72GB (sm_120) with Diffusers; other hardware/runtimes may differ.
• SageAttention v3 quality is lower; see the table above.
• For research and local evaluation only. Follow the licenses of the base model and the example assets.