# SPI Research Report — Video Backends (Phase 2 selection)

Date: 2026-09-23 · Mode: ultra_low (narrow research: 3 candidates + VBench) · Sources: official HF model cards, diffusers docs, vendor repos.

Selection policy (from project brief): openness > controllability > architecture access > fine-tunability > ecosystem > quality > compute efficiency.

## Comparison

| | **Wan 2.1/2.2 (Wan-AI)** | **LTX-2 / LTX-2.5 (Lightricks)** | **Fallback: CogVideoX (zai-org)** |
|---|---|---|---|
| License | **Apache-2.0** (all Wan2.1/2.2 cards) | LTX Open Weights License (`license: other`, gated=auto): free for entities < $10M annual revenue; paid license required for Commercial Entities (≥$10M) on revenue-generating use or commercial fine-tuning | 2b: Apache-2.0; 5b: custom (restrictive) |
| Params | Wan2.2 A14B: 27B MoE (two 14B experts); TI2V-5B dense | LTX-2/2.5: 19B (13B video DiT + audio stream); legacy LTX-Video 2B/13B | 2B / 5.57B |
| Min VRAM (low-res) | Wan2.1 T2V-1.3B: **8.2 GB** (official); TI2V-5B: 24 GB (RTX 4090, 720p@24fps); A14B: ~80 GB native, less with offload/FP8 | LTX-Video 2B: ~10 GB (offloading/fp8); LTX-2 19B: bf16 48 GB+, fp8 ~24 GB | 2b: ~5 GB; 5b: ~10–12 GB (offload) |
| Diffusers pipelines | `WanPipeline`, `WanImageToVideoPipeline`, `WanVACEPipeline`, `WanAnimatePipeline` (stable ≥0.40.0) | `LTXPipeline`, `LTXConditionPipeline`, `LTXLatentUpsamplePipeline`; **`LTX2Pipeline` in v0.40.0** (`diffusers.pipelines.ltx2`) | `CogVideoXPipeline`, `CogVideoXImageToVideoPipeline` (mature) |
| Conditioning | Broadest: T2V, I2V, FLF2V (first/last frame), **VACE** (pose/depth/mask/reference/inpaint/outpaint/video-extend in one pipeline), Animate, S2V; alibaba-pai Fun-Control (pose/depth/trajectory/camera) | T2V, I2V, video-to-video/extend via `LTXConditionPipeline` (`decode_timestep=0.05`); official IC-LoRAs (Ingredients, Clean-Plate, Union-Control, Slow-Motion, upscalers); no ControlNet-style adapter | T2V, I2V; ControlNet only via community CogVideoX-Fun; no official video-extend |
| Fine-tuning | LoRA + full: DiffSynth-Studio (official), community diffusers LoRA | LoRA-first: official distilled/IC LoRAs; `LTX2LoraLoaderMixin` | LoRA via diffusers examples; full FT hard |
| Long video | Wan-VAE handles arbitrary-length 1080p; I2V chaining; 81-frame clips | LTX-Video 0.9.8+ long video; LTX-2 multiscale 2-stage; ~18× faster than Wan2.2 on H100 (1.22 vs 22.3 s/step, 121f 720p) | 49-frame fixed; weakest |
| Ecosystem | Largest (ComfyUI, DiffSynth, Wan2GP, VBench-topping) | Fast-moving (ComfyUI/Kijai, GGUF); gated HF repos add friction | Stale (last update Nov 2024) |

## Recommendation

**Phase 2 primary: Wan 2.2** (+ Wan 2.1 VACE for control jobs). Reasons:
1. Only candidate that is genuinely Apache-2.0 with no revenue clause — matches openness priority.
2. Deepest conditioning surface: one `WanVACEPipeline` covers pose/depth/mask/reference/extend, mapping 1:1 onto SPI's common conditioning API.
3. Official diffusers pipelines for every task family; mock interface translates directly.
4. Cheap lanes for tests (1.3B/5B) and quality lane (A14B MoE).

**Secondary (speed): LTX-2/2.5** — ~18× faster per step, native audio+video stream. Constraints recorded in registry: gated HF access (token with accepted license needed) and the $10M-revenue commercial cap.

**Fallback only: CogVideoX-2b** (Apache-2.0, stale). HunyuanVideo rejected: custom license bans EU/UK/KR use, 13B needs 45+ GB without offload, no first-class diffusers pipeline.

## Known gaps / caveats

- Wan VACE is Wan2.1-based; no official Wan2.2+VACE in diffusers yet → route control jobs to Wan2.1-VACE, quality jobs to Wan2.2.
- LTX-2.x repos are gated → renderer adapter must verify token/license at load time.
- Wan LoRA training official path is DiffSynth-Studio, not diffusers Trainer.
- Pipeline names drift fast; pin `diffusers>=0.40.0` and re-verify `model_index.json` tags at install time.

## VBench / VBench-2.0 (evaluation reference)

- **VBench**: pip `vbench` (0.1.5, Apache-2.0), 16 dimensions. Programmatic: `from vbench import VBench; VBench(device, info_json, out_dir).evaluate(videos_path=..., name=..., dimension_list=[...])`. Needs detectron2 from git for some dims.
- **VBench-2.0**: pip `vbench2` (Apache-2.0, Apr 2025), 18 dimensions (adds physics, human interaction, commonsense). Heavy install (torch 2.5.1 pins, flash-attn, mmcv 2.2.0). Plan: VBench v1 as cheap CI regression gate; vbench2 reserved for release-level evals. SPI uses them as adapters, not as the whole DirectorBench.