Ami1ne/wan1.3b-bora-quant

Wan2.1-T2V-1.3B, quantized to Bora's size

Part 2 of the Bora video (@Codeically): take an existing video model and shrink its transformer to the size of Bora V1 (336M params, 673 MB bf16).

• Base: Wan-AI/Wan2.1-T2V-1.3B-Diffusers, transformer 1.42B params, 2.8 GB bf16
• Weight-only quantization of every linear layer in the 30 transformer blocks (2.5-4.25 bits per weight incl. group scales), methods RTN, HQQ, GPTQ (calibrated on Bora latents)
• models/wan1.3b_gptq_w3g128_ft.safetensors: GPTQ W3 g128, then QA-LoRA adapters + learnable group scales trained 2000 steps on Ami1ne/Bora (256x256, 8 fps) distilled from the bf16 model, and merged back into the quantized weights (same size as before fine-tuning)

Results

tag	transformer_MB	gpu_weights_GB	peak_GB_1clip_cfg	s_per_clip	steps	val_t0.25	val_t0.5	val_t0.75	val_t0.9	val_loss	s_per_clip_8steps
bora_v1	673.091	0.9768	1.0357	0.8276	8	0.2971	0.1713	0.1495	0.1936	0.2029	nan
wan_bf16	2844.25	2.9997	3.2545	4.2222	30	0.2279	0.1487	0.1562	0.2348	0.1919	1.1276
gptq_w3g128	625.773	0.9335	1.2311	5.8692	30	0.2553	0.1507	0.1417	0.2012	0.1872	1.5668
gptq_w3g128_ft	625.773	0.9335	1.2311	5.8955	30	0.2234	0.1347	0.1302	0.1852	0.1684	1.5733

Text encoders are not in these numbers: Bora uses T5 v1.1 base (110M params, 0.4 GB fp32), Wan uses umT5-XXL (5.7B params, 11 GB bf16). Both run once per prompt and can sit on the CPU. The VAE is the same for both.

Use

import wanq                                   # code/wanq.py
model = wanq.load_quantized("models/wan1.3b_gptq_w3g128_ft.safetensors", "cuda")
lat = wanq.wan_sample(model, text, null_text, frames=10, height=32, width=32, steps=30, cfg=5.0, shift=3.0)
# text / null_text: umT5-XXL states padded to 512 tokens (see the Wan pipeline); decode lat with the Wan2.1 VAE:
# video = vae.decode(lat * latents_std + latents_mean)