wan2.2-i2v-a14b-p100a

Wan 2.2 image-to-video A14B with the lightx2v 4-step LoRA, ported to one Tenstorrent Blackhole p100a. Weights: Comfy-Org/Wan_2.2_ComfyUI_Repackaged (fp8) + lightx2v 4-step LoRAs · Port: SunandBean/tt-wan22

Runs on p100a (mesh 1×1, fabric FABRIC_1D). Both experts and the Wan 2.1 VAE encoder and decoder run on the card; the UMT5-XXL prompt encoder runs on the host.

What this port adds — both 14B experts on a single 29.9 GB card by swapping them in and out, a streamed VAE encode and decode so 81 frames never sit on the card at once, the UMT5-XXL host encoder, and the segment renderer. What it builds on — tt-metal's tt_dit WanEncoder / WanDecoder, imported from the runtime rather than vendored (Apache-2.0), and the fp8 + LoRA loader from the sibling tt-qwen-image-edit port.

A 480p 81-frame segment takes 96 s in steady state against 84 s for the same segment on an RTX 5070 Ti.

What this port follows

The reference is the ComfyUI wan_segment graph that the same weight files run on the GPU:

• wan2.2_i2v_high_noise_14B_fp8_scaled for steps 0–1, wan2.2_i2v_low_noise_14B_fp8_scaled for steps 2–3
• lightx2v 4-step LoRAs at strength 1, merged at load (diffusion_model. key prefix)
• shift 5, simple sigmas, Euler, cfg 1 — 4 steps total
• WanImageToVideo / WanFirstLastFrameToVideo conditioning; each segment after the first starts from the previous segment's last frame, as on the GPU

Each expert is 14.3 B parameters in fp8. The model is 40 blocks at dim 5120 (40 heads × 128), FFN 13824 with GELU-tanh. Wan 2.2 I2V has no image cross-attention (k_img); the start frame arrives as 36 input channels (16 noise + 4 mask + 16 conditioning latent). Self-attention q/k RMSNorm is over the whole 5120, not per head, and the 3D rope is 44/42/42 — not the 16/56/56 you might expect.

Quickstart

Prerequisite: a tt-metal / ttnn environment that includes models.tt_dit with the Wan VAE and its conv3d kernels. The tt-inference-server 0.18 image carries one; deploy/run.sh in the GitHub repo is a wrapper that runs the renderer inside it.

hf download sunandbean/wan2.2-i2v-a14b-p100a --include "code/*" --local-dir wan2.2-i2v-a14b-p100a && cd wan2.2-i2v-a14b-p100a
pip install -e code/

Weights are not redistributed. Point COMFY_MODELS at a ComfyUI models directory holding:

$COMFY_MODELS/
  diffusion_models/wan2.2_i2v_{high,low}_noise_14B_fp8_scaled.safetensors
  loras/wan2.2_i2v_lightx2v_4steps_lora_v1_{high,low}_noise.safetensors
  vae/wan_2.1_vae.safetensors
  text_encoders/umt5_xxl_fp8_e4m3fn_scaled.safetensors

All of them are in Comfy-Org/Wan_2.2_ComfyUI_Repackaged under split_files/.

As a batch renderer

JOB_DIR=<job dir> COMFY_MODELS=<ComfyUI models dir> WAN_CACHE=<cache dir> \
  python -m tt_wan22.render

<job dir>/job.json describes the whole video:

{"gen_w": 832, "gen_h": 480, "fps": 16,
 "segments": [{"prompt": "...", "length": 81, "seed": 42, "start": "/images/first.png", "end": null},
              {"prompt": "...", "length": 81, "seed": 43, "start": null, "end": null}]}

start: null continues from the previous segment's last frame. Per segment the renderer writes i.frames.rgb (raw uint8 RGB), i_last.png and i.json (timings); progress.json follows the run and result.json ends it.

As a library

from tt_wan22 import Renderer, open_dev, close_dev

dev = open_dev()
try:
    r = Renderer(dev, progress=lambda **kw: None, swap=False)   # swap=True above ~40k tokens (720p)
    frames, timing = r.segment(prompt, start, end, 832, 480, length=81, seed=42, progress=lambda **kw: None)
finally:
    close_dev(dev)

start and end are [H, W, 3] float tensors in [0, 1], or None.

Demo

One 480p 81-frame segment, same start image, prompt and seed on both devices. Prompt: "The lighthouse beam sweeps across the sky while waves crash against the dark rocks below, the camera slowly pushes in, golden hour light."

Top row: p100a. Bottom row: ComfyUI on an RTX 5070 Ti. Five frames across the segment. The composition, the camera push and the timing of the wave hitting the rock are the same; the spray detail differs. The full segment is at media/segment_p100a.mp4.

Performance

480p (832×480) and 720p (1280×720), 81-frame segments, 4 steps, measured 2026-10-04 on a p100a with the converted-weight cache and the compiled-kernel cache warm.

480p, steady state

Stage	First segment	Steady state
VAE encode, 81 frames (card)	58.8 s	20.7 s
Text K/V (host, float32)	16.3 s	0 s (cached per prompt)
Sampling, 4 steps (card)	85.3 s	61.3 s (15.3 s per step)
VAE decode, 81 frames (card)	22.5 s	13.3 s
Segment total	167.1 s	95.9 s
Same segment, RTX 5070 Ti		84 s

From the published container

The same image anyone can tt-model pull, reached over its own HTTP server on an idle card (2026-10-09). Three 832×480 81-frame segments, seed 42, from a cold start:

Stage	First segment	Second	Third
VAE encode	58.5 s	20.7 s	20.7 s
Text K/V	25.6 s	0 s (cached)	0 s
Sampling, 4 steps	68.1 s	60.5 s	60.4 s
VAE decode	22.4 s	13.2 s	13.2 s
Segment total	175.3 s	95.0 s	94.8 s

Load to ready was 210 s with both experts already converted in the weight cache. The steady-state figure reproduces the 95.9 s measured through the renderer, so the HTTP server costs nothing measurable over the batch path.

720p

First segment (includes compilation at that size)	394 s
Next segment	347 s — encode 47 s, sampling 268 s (including two expert swaps, 24 s), decode 30 s
Per step	~61 s (4× the 480p step: attention is quadratic in length)
Same segment, RTX 5070 Ti	~312 s

At 720p the sequence is 75,600 tokens and the VAE encoder no longer fits beside both experts, so the renderer holds one expert at a time and swaps at step 2 from the weight cache (12 s each). Below ~40,000 tokens both experts stay resident and there is no swap.

Start-up

Load, cold (fp8 dequantise + LoRA merge + tile conversion)	348 s — 157 s per expert
Load, from the converted-weight cache	46 s — 12.2 s per expert
Load, with the kernel cache warm as well	30 s
Converted-weight cache on disk	9.4 GB per expert (884 tensors)

The weight cache (~/.cache/wan-p100a, WAN_CACHE) keys on the checkpoint and LoRA file sizes and mtimes, the precision settings and the tt-metal build, and is only marked complete once a full load finishes. Output is bit-identical with and without it (latent PCC 1.0). TT_METAL_CACHE keeps the compiled kernels: without it every new container recompiles (720/720 misses, about 75 s on the first segment).

Accuracy

Against ComfyUI on an RTX 5070 Ti with the same input and seed, 480p 81 frames:

Final latent PCC	0.978–0.981
PSNR, frame 0	39.0–39.4 dB
PSNR, frame 40	25.3–26.2 dB
PSNR, frame 80	24.3–24.8 dB

Composition, motion (including the timing of waves hitting rocks) and colour match; the spray detail differs. The per-latent-frame PCC falls from 0.998 at the front to 0.955 at the back, which is what the falling PSNR reflects. The VAE decoder alone, run against the CPU reference, is PSNR 51 dB, and the conditioning encode is PCC 0.99998.

How it was made to fit

The card has about 30 GB and two fp8 experts are 23.3 GB converted. Three things made the rest fit:

1. Cross-attention K/V weights stay on the host. Together they are about 4.3 GB and are used exactly once per prompt, to build the text K/V. Computing that on the host in float32 and uploading only the K/V drops device residency from 23.3 GB to 19.0 GB — enough for the decoder.
2. Streamed VAE. tt_dit's encoder uploads all 81 frames (4.1 GB) and slices on the card; encode_streamed sends frame 0 then 4 frames at a time. The decoder gathers all 81 output frames (about 2 GB); decode_streamed brings each piece back to the host.
3. Padded keys are truncated, not masked. The 32,760 → 32,768 padding would need a 32768² bf16 mask — 2 GB, which OOMs beside the experts. Passing K/V at the logical length 32,760 gives the same result (sdpa_pad_check.py: PCC 0.99978 against the mask, 0.99763 with no mask at all).

Tuning

Profiling a 480p step showed SDPA 42%, reshape + permute 31% (head splitting) and matmul 18%.

• Head splitting: one concat + nlp_create_qkv_heads instead of per-tensor reshape + permute — 0.129 s → 0.0115 s, identical output.
• SDPA: already near the operation limit (~110 TFLOPS), so HiFi4 → HiFi2 with fp32 accumulation bought 9% (PCC 0.99974). Query chunks of 128 or less are 2× slower; LoFi drops PCC to 0.82–0.94 and is unusable.
• Step time: 22.2 s → 15.3 s.

The 5120² bfp8 matmuls already run at about 260 TFLOPS. 62% of sampling is still self-attention SDPA.

Caveats

• Single card only. Mesh 1×1. tt-metal's own tt_dit Wan pipeline supports only 4–8 chip Blackhole meshes, which is why one expert was rewritten for a single chip.
• 480p and 720p only, at the model's native 16 fps. 1080p upscaling and RIFE frame interpolation are ComfyUI graph nodes and are not part of this port.
• Not bit-identical to the GPU. PSNR 24–39 dB depending on the frame; see Accuracy. Use it for the same shot, not to reproduce an existing GPU render exactly.
• Segments are sequential by construction — each starts from the previous one's last frame — so this port does not make a single video faster. Against the GPU it is roughly at parity per segment; its value is running videos while the GPU does something else.
• A render holds the card for minutes at a time. On a shared card that blocks everything else.
• FFN weights are bfp4. The notes record this as a quality risk that was checked and not observed; it is still a lower precision than the GPU path uses.
• The UMT5 prompt encoder is still on the host (about 14 s of CPU per new prompt; cached per prompt afterwards).

Licensing

• Weights: Comfy-Org/Wan_2.2_ComfyUI_Repackaged (fp8 repackagings of Wan-AI/Wan2.2-I2V-A14B) and the lightx2v 4-step LoRAs — Apache-2.0. Not redistributed here.
• Port code (code/): Apache-2.0 (SPDX headers on the modules), from SunandBean/tt-wan22.
• The Wan VAE on the card uses models.tt_dit's WanEncoder / WanDecoder from tt-metal (Apache-2.0), imported from the runtime rather than vendored.
• The fp8 + LoRA checkpoint loader (comfy_weights.py) is shared with the tt-qwen-image-edit port.
• code/THIRD-PARTY.md lists every directory under code/ that came from somewhere else, with its upstream commit and where its licence text sits in this repo.

Run it as a container

Everything below is generated by tt-model package from the manifest this image was built from.

tt-model pull  sunandbean/wan2.2-i2v-a14b-p100a --with-weights
tt-model serve sunandbean/wan2.2-i2v-a14b-p100a

pull --with-weights downloads the Docker image and the Comfy-Org/Wan_2.2_ComfyUI_Repackaged weights (into your HF cache; they are not in the image). serve starts the model's own HTTP server on port 20000 (or the next free port, if that one is busy); the first start compiles kernels for your device, which takes several minutes, and the server is ready when it logs Application startup complete.

Request shape

POST /predict takes {"prompt": str, "start_image": b64, "end_image": b64 | null, "width": int, "height": int, "length": int, "seed": int} and returns the segment's raw uint8 RGB frames as frames_b64 with shape, a last_frame PNG and timing_s.

One segment at a time -- an image plus a prompt in, a short clip out. Chain segments by feeding last_frame back as the next start_image. length must be 4n+1 (the VAE's temporal stride) and the LoRA fixes the schedule at 4 steps; anything else is a 422.

The envelope (832x480, 81 frames by default) is fixed at startup because it decides whether both experts stay resident on the card or swap between steps. Widen it with WAN_MAX_W, WAN_MAX_H and WAN_MAX_FRAMES before starting the server, not per request.

Provenance

The exact sources the image was built from — the model code under code/ is byte-identical to what is inside the image (pyproject.toml, PYTHON.md and examples/ are packaging files this repo adds, and are not in the image):

component	built from
tt-metal	58648d44443223041148125843ca7c7d4780463e
code/ digest	e7606c1125810eb8 (sha256, first 16 hex digits)
built	2026-10-10T02:47:59+00:00 by tt-model 0.1.0