Spaces:
Running
Running
Trim org card
Browse files
README.md
CHANGED
|
@@ -20,43 +20,3 @@ pinned: false
|
|
| 20 |
</div>
|
| 21 |
|
| 22 |
**Veda** is a learned sparse-attention method for video DiTs. A lightweight tile-score predictor, distilled from full attention, keeps only the attention tiles that matter and skips the rest with a hardware-aligned tile-sparse kernel.
|
| 23 |
-
|
| 24 |
-
- 🎯 **90% sparsity at perceptual parity** with full attention (WaverBench1.0 human side-by-side).
|
| 25 |
-
- 🚀 **95% sparse 720P generation: 19.4 min → 3.8 min** — **5.1× end-to-end** and **10.5× self-attention** acceleration vs. FlashAttention-3 (Waver-T2V-12B, 720P, 241 frames).
|
| 26 |
-
- 📈 **Near-linear scaling**: at 245K tokens, per-layer latency drops from 1576 ms to 309 ms.
|
| 27 |
-
- 🔌 **Plug-and-play**: Veda only changes how attention is computed. Backbone weights and style stay untouched, so it works with LoRAs and fine-tuned variants.
|
| 28 |
-
|
| 29 |
-
<table>
|
| 30 |
-
<tr>
|
| 31 |
-
<td align="center"><img src="https://veda-sparse.github.io/static/posters/race/race_full.webp" width="100%"/><br/><sub>FlashAttention-3 · full attention · 19.4 min</sub></td>
|
| 32 |
-
<td align="center"><img src="https://veda-sparse.github.io/static/posters/race/race_veda.webp" width="100%"/><br/><sub><b>Veda</b> · 95% sparsity · 3.8 min</sub></td>
|
| 33 |
-
</tr>
|
| 34 |
-
</table>
|
| 35 |
-
|
| 36 |
-
## How it works
|
| 37 |
-
|
| 38 |
-
1. **Full-attention structure distillation.** Max-pooled full attention serves as a tile-level teacher; a KL objective aligns the predictor with the dense attention geometry before Top-k selection.
|
| 39 |
-
2. **Head-conditioned tile geometry.** Per-layer, per-head tile shapes (p<sub>t</sub>, p<sub>h</sub>, p<sub>w</sub>) adapt to heterogeneous spatial and temporal heads under a fixed hardware budget.
|
| 40 |
-
3. **Tile-sparse kernel.** A ThunderKittens kernel fetches only the selected K/V tiles via TMA and overlaps sparse gathers with WGMMA compute, turning sparsity into wall-clock speedup.
|
| 41 |
-
|
| 42 |
-
## Releases
|
| 43 |
-
|
| 44 |
-
| | Description |
|
| 45 |
-
|---|---|
|
| 46 |
-
| 🤗 [Minimax-H3-T2VA-Veda-8NFE-600Step-Preview](https://huggingface.co/Veda-Sparse/Minimax-H3-T2VA-Veda-8NFE-600Step-Preview) | Veda predictor for MiniMax-H3 text-to-audio-video generation (8-step, with Turbo LoRA) |
|
| 47 |
-
| 💻 [Miowtion](https://github.com/veda-sparse/Miowtion) | Sparse acceleration and LoRA fine-tuning of MiniMax-H3 on resource-constrained machines |
|
| 48 |
-
| 💻 [Veda-on-ComfyUI](https://github.com/veda-sparse/Veda-on-ComfyUI) | ComfyUI integration |
|
| 49 |
-
| 💻 [vllm-omni](https://github.com/veda-sparse/vllm-omni) | Efficient inference for omni-modality models |
|
| 50 |
-
|
| 51 |
-
## Citation
|
| 52 |
-
|
| 53 |
-
```bibtex
|
| 54 |
-
@inproceedings{han2026veda,
|
| 55 |
-
title = {Veda: Scalable Video Diffusion via Distilled Sparse Attention},
|
| 56 |
-
author = {Han, Shihao and Yang, Hao and Mei, Xiaofeng and Hu, Xinting and Jiang, Yi and Qi, Xiaojuan},
|
| 57 |
-
booktitle = {International Conference on Machine Learning (ICML)},
|
| 58 |
-
year = {2026}
|
| 59 |
-
}
|
| 60 |
-
```
|
| 61 |
-
|
| 62 |
-
<sub>ByteDance · The University of Hong Kong · USTC</sub>
|
|
|
|
| 20 |
</div>
|
| 21 |
|
| 22 |
**Veda** is a learned sparse-attention method for video DiTs. A lightweight tile-score predictor, distilled from full attention, keeps only the attention tiles that matter and skips the rest with a hardware-aligned tile-sparse kernel.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|