Three of ten optimisations did nothing: video generation on one card

Two self-hosted video models, five configurations, one card each. Generation time came down by 2.1x to 5.9x with no weight conversion and no change to the API contract.

The part worth reading is not the speedups. It is which knobs did nothing: three of the ten things tried on Wan2.2 t2v returned zero, and fp8 quantisation that gave 1.5x on t2v gave nothing at all on s2v.

Setup

The Wan2.2 work replaced an in-house single-file server with vLLM-Omni behind an adapter that keeps the old /v1/video/generations contract byte for byte. The H3 work changed neither the model nor the container image: it swapped one build of the same Hugging Face repo for another and attached the vendor's Turbo LoRA.

How the numbers were taken

Wall clock from request to complete video. Server-side timers read 1 to 2 seconds lower, which is the mp4 mux plus base64 encode. Fixed prompt, fixed seed, requests serial, median of three. vLLM-Omni compiles transformer blocks lazily, so every configuration got one warm-up request that was thrown away; without it you are measuring compilation, not inference.

Two parameters are easy to misread. fps does not change generation time. Diffusion produces a fixed frame count and fps only sets playback speed: 49 frames is a 3.1 s clip at fps 16 and an 8.2 s slow-motion clip at fps 6, for the same cost. And s2v's width/height are an area budget rather than an output size — the real dimensions follow the reference image's aspect ratio, so a request for 720x1280 against an 832x1104 reference returns 832x1088.

What each step bought

Grey bars that match the length of the bar above them are the interesting ones.

MiniMax H3

The 8-step LoRA takes the step count from 20 to 8. Per-step cost did not move: 8.06 s before, 8.08 s after. The entire 2.24x is the twelve steps not taken, and none of it is the LoRA being cheaper to evaluate.

The weight swap is where per-step cost changed, 8.08 s to 3.49 s, worth 1.91x by itself. Same repo, same 19.5 GiB file size, no image rebuild, no CUDA change. int8_convrot was the slower of the two builds on this card, and it is the one the bundled template ships with.

Peak memory went the other way: 45.8 GiB on int8, 61.9 GiB on fp8, against 95.6 GiB on the card. We did not chase the reason. On a single-tenant 96 GB card the memory was spare and the speed was not, so the trade was easy — but that is a property of this deployment, not a general result.

Wan2.2 text to video

The baseline is not a one-off: four historical production calls logged 131.6 to 132.0 s before the re-measurement at 132.3 s.

A later change fused LightX2V's 4-step distillation LoRA into both experts — the 10.7 s row in the headline table. A control run at the old parameters (20 steps, CFG 4.0) came back at 66.4 s, level with the earlier result, so the distillation LoRA does not change per-step cost either. The 5.86x comes from being able to drop to 4 steps and CFG 1.0, and the CFG half matters: distillation bakes guidance into the weights, so CFG above 1.0 spends a whole unconditional forward pass for nothing.

Wan2.2 speech to video

cache_dit is the largest single win anywhere in this work: 2.07x on s2v against 1.23x on t2v. fp8, worth 1.29x on t2v, returned 0.6% here and left memory unchanged — which says it is not reaching the s2v transformer at all.

Where the time goes

Sampling was the only stage this round touched, and after it shrank 5.8x the fixed work became a third of the total. That is the ceiling on further sampling work: even free sampling would only reach about 13 s.

SaveVideo is libx264 on the CPU. PyAV inside the container can construct an h264_nvenc encoder, but ComfyUI's SaveVideo node does not expose the codec, so using it needs a patch or a custom node.

Clip length

Longer clips cost more per second of output, because attention is quadratic in sequence length. Measured on H3 at 8 steps and fp8:

2.86x the frames costs 3.78x the time. Frame counts have to land on H3's 17k+5 grid (73, 124, 175, 209, 362); anything else is rounded up.

What did not work

The memory assumption was wrong. Going in we expected the memory switches to help. The GPU sits at 100% utilisation and 598 W against the power cap for the whole generation, so compute was the limit and trading compute for memory could only ever cost time.

Sequence parallel does work, and we did not ship it. Four cards take t2v to 32.0 s, but output per card falls from 1.63 to 1.03 relative to the single-card baseline, while four independent single-card replicas emit a clip every 15.8 s — 2.03x the throughput. Multi-GPU buys single-clip latency with aggregate throughput. Whether that trade is worth making depends on whether one user is waiting or a batch is being produced.

Limitations

• Quality has not been evaluated systematically. All that was done is a same-seed frame comparison by eye, which showed no visible artefacts. cache_dit is lossy by construction, so anything running on it long term needs per-frame evaluation, and the H3 8-step distillation deserves the same.
• The H3 numbers were taken with --enable-triton-backend, worth 3.7% of sampling time and 1.8% end to end. Inside the rounding here, but it was present.
• The first call after any restart takes roughly twice the steady-state time. Transformer blocks compile lazily and the first request pays for it. It is not a hang.
• Peak memory on H3 fp8 is 16 GiB higher than int8 and we did not find out why.

Raw data: video.csv · video-techniques.csv · h3-stage-breakdown.csv · h3-length-scaling.csv

---

MIT licensed, reusable with attribution. Wan2.2 is released by the Wan team at Alibaba; MiniMax H3 by MiniMax, with ComfyUI-format weights and the 8-step Turbo LoRA from Comfy-Org/MiniMax-H3. The 4-step distillation LoRA is LightX2V's. Full methodology at github.com/ecohash-ai/ecohash-benchmarks. $1 of free credit on a new account — console.ecohash.com.