SlimKV: Joint Token-Feature KV Cache Compression with Reconstruction-Free Beacon Attention
Paper • 2610.02953 • Published
YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
SlimKV
Joint token-feature KV-cache compression with reconstruction-free beacon attention.
The r2/r4/r8 suffixes denote 2×/4×/8× feature compression (SlimKV-f2/f4/f8 in the paper). Token compression is configured separately through the beacon interval. In the paper, t4f4 combines 4× token compression and 4× feature compression for 16× total compression.
| Backbone | 2× feature compression | 4× feature compression | 8× feature compression |
|---|---|---|---|
| Llama3.1-8B | SlimKV-f2 (r2) | SlimKV-f4 (r4) | SlimKV-f8 (r8) |
| Qwen2.5-14B | SlimKV-f2 (r2) | SlimKV-f4 (r4) | SlimKV-f8 (r8) |
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support