Commit History
-
initial release: H2O KV cache eviction for DeepseekV3 / MLA architectures
GENOMA LABS / researchReference implementation of importance-based KV cache eviction (H2O, Zhang et al. 2023) patched onto DeepseekV3Attention layers. Compatible with DeepSeek V3 / V3.2, Kimi K2 / K2.6, and any future MLA-based model that subclasses the same attention layer. Memory model: 82 GB -> 10.2 GB at budget=4096 on canonical 61-layer DeepseekV3 layout. Roughly 2-4x prefill speedup at 128K-1M context with no retraining and no architectural changes. Contents: - src/kv_eviction_mla.py H2O patch + smoke test (277 LOC) - notebooks/01_smoke_test_walkthrough.md annotated explanation + memory model - docs/HOW_IT_WORKS.md where eviction hooks into the layer + porting notes - LICENSE Apache 2.0 - README.md quick start + budget guidance Companion to GENOMA LABS / research handbook on sub-quadratic attention (github.com/genoma-labs/subquadratic-research, public on flip).a8d4591