Combined VACE Self-Forcing — four-step checkpoints
Four milestones from the same combined VACE Self-Forcing run, each following 1,000 causal flow-matching warm-start updates. The default root student.safetensors is SF5000, identical to the weights in checkpoints/5000/.
| SF outer iteration | SF student updates | Checkpoint folder |
|---|---|---|
| 2,000 | 400 | checkpoints/2000 |
| 3,000 | 600 | checkpoints/3000 |
| 4,000 | 800 | checkpoints/4000 |
| 5,000 | 1,000 | checkpoints/5000 |
Each folder contains student.safetensors, training_summary.json and export_verification.json. The loader, runtime code, architecture config and empty text context are shared at the repository root. Root metadata describes SF5000; use the selected folder's records for another milestone.
These custom Fixer/FastGen weights repair an edited draft video through normalized Wan VAE latents. The complete student has 1,264 tensors, covering its main backbone and VACE branch, with original FP32 values and transformer.* names preserved. This is an inference package, without the optimizer states, fake-score model, discriminator or teacher weights needed to resume training. No demonstration videos are included in this update; previously shared SF2000 videos describe that checkpoint only.
Download and load one checkpoint
Use FastGen commit c40fbea17b9a28a38528ee5254471dac4f2f9f12, PyTorch 2.8.0, Diffusers 0.35.1, Transformers 4.49.0 and PEFT 0.17.1. Install the pinned FastGen dependencies, including safetensors and huggingface_hub, and put the FastGen checkout on PYTHONPATH. CUDA is required for sampling.
Use allow_patterns to download only the shared files and selected weights, instead of every milestone. For SF3000:
import sys
import torch
from huggingface_hub import snapshot_download
common = [
"README.md", "load_student.py", "inference_config.json",
"empty_prompt_context.pt", "runtime/**",
"Wan2.1-VACE-1.3B-diffusers/transformer/config.json",
]
folder = snapshot_download(
"primecai/fixer-combined-vace-self-forcing",
allow_patterns=common + ["checkpoints/3000/*"],
)
sys.path.insert(0, folder)
from load_student import load_student, repair_latents
student = load_student(folder, checkpoint=3000) # CUDA, bfloat16
torch.backends.cuda.matmul.allow_tf32 = False
torch.backends.cudnn.allow_tf32 = False
# User-supplied normalized Wan VAE output tensors, with identical shapes:
# draft: edited video; inactive: all-zero normalized RGB video encoded by VAE.
# An 81-frame 832x480 video gives [B,16,21,60,104] for both.
draft = torch.load("draft_latents.pt", map_location="cpu", weights_only=True)
inactive = torch.load("inactive_latents.pt", map_location="cpu", weights_only=True)
repaired = repair_latents(student, draft, inactive, folder, seed=42000)
# Decode repaired with the same original Wan VAE.
To load another milestone, change both checkpoints/3000/* and checkpoint=3000 to the desired iteration. Pass the repository root as folder, not the checkpoint subdirectory.
For the default SF5000 root weights, use the same common patterns with:
folder = snapshot_download(
"primecai/fixer-combined-vace-self-forcing",
allow_patterns=common + [
"student.safetensors", "training_summary.json", "export_verification.json"
],
)
student = load_student(folder)
CPU verification is supported with load_student(folder, device="cpu", dtype=torch.float32, checkpoint=3000). Loading is strict and uses no pretrained-weight fallback. Keep the supplied Wan2.1-VACE-1.3B-diffusers directory name: pinned FastGen identifies the model family from that path.
Conditioning and sampling
Encode the full edited RGB draft with the original Wan VAE. Separately VAE-encode an all-zero RGB tensor in normalized [-1,1] space, with the same frame count and geometry, to obtain inactive. It is not a tensor of latent zeros. Both inputs have shape [B,16,T,H,W] and are already normalized; do not normalize them again.
The helper builds [inactive16, draft16, ones64] as the 96-channel VACE condition and uses empty_prompt_context.pt["context"]. There is no depth extraction or prompt encoding. The edit instruction belongs to the upstream draft editor; the fixer uses empty text and CFG 1.
Retain the edited draft's first RGB frame and encoded latent. Do not replace it with the original source frame, splice in a target latent, or clamp generated RGB frame zero. Preserve each video's frame count and orientation. The helper sets the temporal cache capacity for each input and clears both main and VACE caches before and after sampling.
Sampling uses one latent frame per causal chunk, four SDE denoises at [0.999,0.937,0.833,0.624,0.0], and one clean KV-cache commit per chunk. VACE scale is 1 and context noise is 0. Initial whole-video noise is generated on CPU in FP32 with seed 42000, then cast to the model device/dtype. The independent SDE seed is 142000 (seed + 100000).
The VAE is a separate dependency. The combined OpenVE workflow uses original DiffSynth commit d5d1d3914cdbcdea8e57865627b794068af4a3fa, tiled VAE size (30,52) and stride (15,26), in a separate reference environment with Torch 2.9.1 and Transformers 4.57.3.
Training provenance
The teacher is the combined-data VACE bidirectional step-10,000 model from lioryariv/tfix-models revision 1105338767192df1c802914d43c337a7269e7e5a, at filtered-combined-v1-20261103/vace-bidirectional/checkpoint-0010000/branch.safetensors. Its Wan2.1-VACE-1.3B base revision is 574e6a744642ce3bee319afc31496b88bde8aac4.
Training uses corrected lioryariv/tfix-combined-data revision a8365564aa548a62316c9d55ec06d77636d2224e, following the combined VACE handoff. The old component's first RGB frame was corrected before encoding the entire draft. Training consumes those stored corrected drafts without target-latent frame-zero replacement. This run replaces the earlier tfix-vace data and step-5873 teacher.
Warm start used 1,000 updates at LR 5e-5. SF uses DMD2 plus GAN at LR 5e-6, with one student update per five outer iterations. The run uses 32 H200 GPUs across four nodes, global batch 64, microbatch 2 and gradient accumulation 1, with activation checkpointing and no EMA.