memset0/vsqa-preview-14b

VSQA preview 14B — DMD step 100

Preview / intermediate checkpoint. Not evaluated. This is the student of an in-progress 1000-step DMD run, exported at step 100. It is published for early inspection only; later checkpoints of the same run may replace it.

What it is

A few-step (3-step DMD) student of Wan2.1-T2V-14B trained with video sparse attention (VSA) and NVFP4 quantization-aware training (QAT) of both attention and linear layers (VSQA project).

Training lineage (all at 77 frames, 1280×768, global batch 16):

1. Dense NVFP4 per-layer reconstruction, 250 steps, from the original Wan2.1-T2V-14B weights.
2. Plain ODE initialization (endpoint regression, no reconstruction loss), 500 steps, VSA C256 cube (4, 8, 8) at sparsity 0.9, NVFP4 attention and linear QAT.
3. DMD distillation (this checkpoint: step 100 of 1000), student LR 2e-6, critic LR 4e-7, generator updated every iteration, no EMA. Teacher and critic are the original dense Wan2.1-T2V-14B.

Sampling contract

• 3 denoising steps at timesteps [1000.0, 941.1763916015625, 800.0], no additional timestep warp.
• flow_shift = 8.0, guidance scale 1.0 (distilled), 77 frames at 1280×768.
• Attention backend used in training: VSA_QAT_TRAIN_C256 (sparsity 0.9); linear quantization nvfp4_qat_train. Reproducing the trained numerics needs the FastVideo VSA/QAT kernels; running the weights with plain dense BF16 attention and linears is a different (untested) configuration.

Files

Diffusers layout. transformer/ holds the exported student; text_encoder/, tokenizer/, vae/ and scheduler/ are unchanged from the base model. metadata.json (normalized training contract), artifact_manifest.json, resolved.yaml, source.yaml and export-verification.json record provenance and the strict-reload export verification.

License follows the base model (Apache-2.0).