WaveCut/Vimoo-1.0-Lite-OrbitQuant-W4A8

Vimoo 1.0 Lite — OrbitQuant W4A8

OrbitQuant 4-bit weights for Vimoo-1.0-Lite, the ≈9.5B Vimoo video model that makes 720p, 24 fps video from text (T2V) or from a first frame and text (I2V). The transformer and the Qwen2.5-VL text encoder both run 8-bit activations over 4-bit weights. The 4-bit codes were chosen with GPTQ rounding against calibration prompts that are not in the evaluation set. The runtime is the official Vimoo modular pipeline of the Vimoo-AI Diffusers fork with fused OrbitQuant kernels, and the ComfyUI-OrbitQuant nodes. The download is 11.0 GB. The original needs 36.8 GB (its transformer, glyph encoder and the Qwen2.5-VL-7B checkpoint).

One RTX 5090 makes a 5-second 1280×720 video (121 frames, 50 steps) in 4 min 36 s (3 min 21 s with INT8 attention) with 20.5 GiB of device memory at the peak. The BF16 original does not fit in the card's 32 GB, with or without model offload. On an RTX PRO 6000 the same video takes 3 min 20 s instead of 9 min 38 s (2.9× faster). The Performance section has the details.

Examples

BF16 original on the left, this release on the right: the same prompt, seed and sampler, 1280×720, 61 frames (2.5 s at 24 fps), 50 steps. These are evaluation cases; none of them was used for calibration. The image-to-video first frames come from the Diffusers documentation images (huggingface/documentation-images).

Text to video, seed 201. A lighthouse on a rocky cliff during a storm at night. Huge waves crash against the rocks and spray rises into the wind. The lighthouse beam sweeps across the dark sea and rain. Lightning briefly lights the clouds. Slow push-in from the sea, dramatic cinematic lighting.

Text to video, seed 203. Close-up of a hand-painted wooden sign outside a cozy cafe that reads "FRESH COFFEE" in white letters. A barista opens the door and steam from a cup drifts past the sign. Morning light, shallow depth of field, gentle handheld movement.

Text to video, seed 204. A red fox trots through deep fresh snow in a quiet pine forest, stops, listens, and pounces headfirst into the snow. Snowflakes fall slowly. Telephoto tracking shot, soft overcast light, nature documentary style.

Image to video, first frame cat.png, seed 213. The black and white cat slowly lifts its head, its ears twitch, and it looks to the left and then back at the camera, blinking calmly. Its whiskers move slightly. Static camera, soft natural light.

Image to video, first frame astronaut.jpg, seed 211. The small astronaut hops forward across the lunar surface, kicking up fine grey dust that drifts slowly in low gravity. The astronaut turns its helmet toward the camera. The camera slowly tracks alongside.

Usage

Use Linux, Python 3.12 and an NVIDIA GPU. The tested stack is Torch 2.12.1 with CUDA 13.0 on an RTX 5090 and an RTX PRO 6000 (Blackwell).

python3.12 -m venv .venv && . .venv/bin/activate
pip install torch==2.12.1 torchvision==0.27.1 --index-url https://download.pytorch.org/whl/cu130
pip install huggingface_hub
hf download WaveCut/Vimoo-1.0-Lite-OrbitQuant-W4A8 --local-dir vimoo-lite-orbitquant
cd vimoo-lite-orbitquant
pip install -r requirements.txt

requirements.txt installs orbitquant 0.17.1 and the Vimoo-AI Diffusers fork at a pinned commit. On the first run the pipeline downloads the Qwen2.5-VL tokenizer, the Wan 2.2 autoencoder and, for image to video, the SigLIP image encoder from their upstream repositories, as the original does. It does not download the Qwen2.5-VL-7B weights: the quantized text encoder is in this repository.

Text to video:

python run_vimoo.py --prompt 'A lighthouse on a rocky cliff during a storm at dusk. Huge waves crash against the rocks and throw white spray into the air, the beam of the lighthouse sweeps through the rain, dark clouds roll across the sky. Wide shot, slow push-in.' --output lighthouse.mp4

Image to video (the image is the first frame):

python run_vimoo.py --image cat.png --prompt 'The cat slowly turns its head toward the camera and blinks. The camera remains fixed throughout.' --output cat.mp4

Python, inside vimoo-lite-orbitquant:

import torch
from vimoo_orbitquant import load_pipeline

pipe = load_pipeline(".", task="t2v")  # task="i2v" for image to video, with reference_image=PIL image
video = pipe(prompt="A prompt ...", **pipe.vimoo_defaults, generator=torch.Generator("cpu").manual_seed(1234),
             output_type="np", output="videos")[0]

load_pipeline returns the official Vimoo ModularPipeline with the W4A8 transformer and text encoder; pipe.vimoo_defaults holds the settings of the official examples. Options of run_vimoo.py (load_pipeline takes the same settings as arguments):

• --offload: keep the models in host memory; each moves to the GPU for its own stage.
• --attention int8: INT8 Q·K attention. It is faster (text to video on an RTX 5090: 3 min 21 s instead of 4 min 36 s) and changes the video slightly; the distance to BF16 stays about the same (LPIPS 0.375 with INT8 attention, 0.368 without; see Quality). It needs orbitquant 0.17.1 or later.
• --vae-tiling auto|on|off: decode the video in tiles. auto (the default) tiles on GPUs with less than 60 GiB: the Wan 2.2 decode of a 1280×720×121 video does not fit next to the models in 32 GB.
• --width, --height, --frames (4k+1), --fps, --steps, --seed, --negative-prompt.

Sampling defaults follow the official Lite examples: 1280×720, 121 frames at 24 fps, 50 steps, flow shift 8, CFG 4 for text to video and CFG 5 with reference guidance 3 for image to video, and the model's negative prompt. The original prompting guide applies: detailed English prompts work best.

ComfyUI

ComfyUI-OrbitQuant 0.10.0 or later lists this release as Vimoo-1.0-Lite. Connect OrbitQuant Model Loader to OrbitQuant Generate Model Video and save the result with the standard Save Video node. Connect one image to reference_images for image to video. The generator runs in the Python environment that python_executable selects. Install the packages of the Usage section there.

• profile: default, model-offload or int8-attention.
• keep_loaded (off by default for video): the worker keeps the model for the next request.

Checked with ComfyUI v0.44.0 on one RTX 5090 (Torch 2.12.1, CUDA 13.0, orbitquant 0.17.1) through the queue and the standard Save Video node, at the default 1280×720 with 121 frames and 50 steps. Device memory is the NVML total and includes the 1.0 GiB that the idle ComfyUI server holds. Times are per request and include loading when the request starts a worker or changes the kept model.

Request	Profile, worker	Device memory peak	Time
Text to video, first request	default, kept	20.4 GiB	285 s
Text to video, next request	default, kept	20.4 GiB	278 s
Image to video, task change (reload)	default, kept	24.5 GiB	417 s
Text to video, profile change	int8-attention, kept	not separated	207 s
Text to video	model-offload, new worker	not separated	301 s

The int8-attention and model-offload requests started while the previous worker still held its memory, so the NVML total does not separate their peak; Performance measures each profile alone. The MP4 of the first request was byte-identical to the output of run_vimoo.py for the same prompt and seed.

Quality

Evaluation: 5 text-to-video prompts and 3 image-to-video cases that are not in the calibration set, at 1280×720 with 61 frames and 50 steps, with the same prompt, seed and sampler as the BF16 original. LPIPS (AlexNet) and PSNR are computed on every 4th frame against the BF16 video.

Variant	Text to video: LPIPS / PSNR	Image to video: LPIPS / PSNR
OrbitQuant W4A8 (this release)	0.368 / 16.6 dB	0.113 / 34.3 dB
OrbitQuant W4A8, INT8 attention	0.375 / 16.5 dB	—
W4A8 transformer, original BF16 text encoder	0.378 / 16.5 dB	0.147 / 32.2 dB

Text to video drifts more than image to video. Over 50 steps a small difference changes the path of the motion: the W4A8 videos keep the scene, the subjects, the sign text and the camera move of the BF16 videos, but people and waves move differently, and LPIPS counts that as a difference. In image to video the first frame anchors the video, and the W4A8 videos stay close to BF16 (34 dB). The 4-bit text encoder is not the source of the drift: with the original BF16 text encoder the distance to BF16 is the same or larger. It takes 4.4 GB instead of the 16.6 GB Qwen2.5-VL-7B download. Frame sheets (start, 1/3, 2/3 and end of every video): text to video, image to video. Both BF16 and W4A8 write "COFEE" on the cafe sign, so that misspelling comes from the original model. The release loader runs the output layer in chunks of 8192 tokens (see Performance). The chunks round the FP32 sums differently (relative difference 1e-6), which changes a sampled video slightly; over the three image-to-video cases the distance to BF16 stays the same (LPIPS 0.118 against 0.113).

Performance

One video per configuration: 1280×720, 121 frames, 50 steps, the official sampler settings; text to video with the lighthouse prompt of the examples, image to video with the astronaut.jpg case. Each configuration runs in its own process: it loads the pipeline, makes a two-step warm-up video of the same shape and then the timed video. The time covers prompt encoding, sampling and VAE decoding, not loading. Device peak is the NVML total (CUDA context included) sampled every 5 ms during the timed video. Host RAM peak is the maximum resident set of the process, loading included. Torch 2.12.1, CUDA 13.0, orbitquant 0.17.1; the W4A8 rows load the release through load_pipeline.

The release loader runs the output layer of the transformer in chunks of 8192 tokens. Every operation in it is per token, but over the whole 1280×720×121 sequence its FP32 temporaries take several GiB: chunking lowered the PyTorch allocator peak of image to video from 29.7 to 22.8 GiB at the same speed.

RTX PRO 6000 Blackwell (96 GB)

Configuration	Text to video	Device peak	Image to video	Device peak	Host RAM peak
BF16 (original)	9 min 38 s	61.2 GiB	14 min 55 s	61.7 GiB	41.2 GiB
BF16 (original), model offload	10 min 06 s	38.9 GiB	15 min 24 s	44.5 GiB	52.1 GiB
OrbitQuant W4A8	3 min 20 s	34.9 GiB	5 min 03 s	36.7 GiB	9.8 GiB
OrbitQuant W4A8, INT8 attention	3 min 00 s	34.9 GiB	—	—	8.2 GiB
OrbitQuant W4A8, model offload	3 min 28 s	26.5 GiB	5 min 11 s	25.7 GiB	25.4 GiB

RTX 5090 (32 GB)

Configuration	Text to video	Device peak	Image to video	Device peak	Host RAM peak
BF16 (original)	out of memory	—	out of memory	—	—
BF16 (original), model offload	out of memory	—	out of memory	—	—
OrbitQuant W4A8	4 min 36 s	20.5 GiB	6 min 51 s	24.2 GiB	9.3 GiB
OrbitQuant W4A8, INT8 attention	3 min 21 s	20.5 GiB	—	—	8.2 GiB
OrbitQuant W4A8, model offload	4 min 47 s	13.5 GiB	7 min 06 s	17.8 GiB	22.7 GiB

The RTX 5090 decodes the video in tiles (--vae-tiling auto); the RTX PRO 6000 decodes it whole, which raises its device peak. The BF16 original runs out of memory on the RTX 5090, resident and with the official model offload. INT8 attention was measured for text to video only.

Components

Component	Precision	Size
Transformer, ≈9.5B parameters	OrbitQuant 4-bit weights with GPTQ rounding and 8-bit activations for the 480 attention and feed-forward projections of the 40 dual-stream blocks. The patch and condition embedders, time and modulation projections, token refiner and output head keep the original precision: BF16, and FP32 where the original stores FP32 (time embedding, modulation and the output projections).	5.71 GB
Text encoder, Qwen2.5-VL-7B text tower	OrbitQuant 4-bit weights with GPTQ rounding and 8-bit activations for the projections of layers 0–25. The pipeline reads the output of layer 25 (hidden_states[-3]), so the loader skips layers 26–27 and the final norm. BF16 token embeddings and norms.	4.36 GB
Glyph encoder (ByT5)	FP32, unchanged; runs on the CPU as in the original	0.88 GB
Autoencoder (Wan 2.2), SigLIP image encoder, tokenizers	Unchanged, downloaded from the upstream repositories	—

GPTQ calibration of the transformer: its inputs at 12 of the 50 steps of BF16 sampling at 1280×720 with 33 frames, for 6 text-to-video prompts and 9 image-to-video cases (3 images, and the first frames of the 6 text-to-video calibration videos with their prompts). Text encoder: 600 prompts (the calibration prompts, the model's negative prompt and PartiPrompts), encoded as the pipeline encodes them. None of them is an evaluation case. With nearest rounding the mean relative output error of the 320 transformer projection groups is 0.0094; GPTQ lowers it to 0.00054 with the same codebook, bit width and file format (text encoder: 0.0059 → 0.00010). transformer/gptq-info.json and text_encoder/gptq-info.json list the error of each group.

Files

• transformer/: the packed transformer (model.safetensors), its OrbitQuant configuration and manifest, the original config.json and gptq-info.json (the error of each projection group before and after GPTQ rounding).
• text_encoder/: the quantized Qwen2.5-VL text tower and its gptq-info.json. text_encoder_2/, tokenizer_2/, scheduler/, guider/, guider_i2v/: the files of the original model.
• vimoo_orbitquant.py: the loader. run_vimoo.py: the command-line entry point. requirements.txt: the runtime pins. The fused transformer blocks come from orbitquant 0.17.
• modular_model_index.json, inference_defaults.json, recipe.json: the original pipeline description.
• manifest.json: the components, the source revision and the runtime pins. SHA256SUMS: the checksums of all files.
• assets/: the chart and videos of this card.

Limits and license

• The fused blocks run one sample at a time; the pipeline's classifier-free guidance batch runs as consecutive samples.
• Tested on Blackwell GPUs (RTX 5090, RTX PRO 6000) with Torch 2.12.1 and CUDA 13.0, at 1280×720×121 and 848×480×49. Other GPUs and sizes were not tested.
• Use orbitquant 0.17.1 or later (requirements.txt pins it). With 0.17.0 the INT8 attention overflowed FP16 on Vimoo-1.0-Lite and returned a black video.
• Text to video does not reproduce the BF16 motion frame by frame (see Quality).

This release is a quantized version of Vimoo-1.0-Lite by Vimoo-AI under the Apache License 2.0 (see LICENSE and THIRD_PARTY_NOTICES.md). The text encoder is quantized from Qwen2.5-VL-7B-Instruct (Apache License 2.0). Vimoo-AI and the Qwen team did not create, endorse or validate this release.