README.md

metadata

title: README
emoji: 🧠
colorFrom: blue
colorTo: indigo
sdk: static
pinned: false

LowRAM

Why spend memory on what you can predict?

LowRAM compresses the KV cache of long-context LLMs. It reconstructs values from their keys, stores corrections for the hardest-to-predict values, and compresses keys across layers. The base model's weights stay frozen, and lightweight predictors adapt to each prompt at test time. LowRAM can be combined with quantisation and token eviction.

📄 Paper · 💻 Code · 🌐 Website

Highlights

• Up to 26.9× KV-cache compression while keeping 94.5% of the uncompressed LongBench score (Mistral-7B-Instruct-v0.3, prompts over 10K tokens, with quantisation and eviction)
• 4× value-cache compression while keeping 99.3% of the uncompressed LongBench score (Llama-3.1-8B-Instruct)
• Up to 5× higher peak decode throughput at 262K-token context (Llama-3.1-8B-Instruct)

These results come from separate experimental settings; see the paper for details.

Checkpoints

Base model	Repository
Llama-3.1-8B-Instruct	lowram-model/lowram-llama-3.1-8b
Mistral-7B-Instruct-v0.3	lowram-model/lowram-mistral-7b

Each repository contains the meta-learned value-MLP initialisation, pretrained value MLPs and the attention predictor for that model.

Citation

@article{lowram2026,
  title   = {{LowRAM}: Low-Rank Keys and Amortised Values for
             Composable {KV} Cache Compression at Test-Time},
  author  = {Delgado, Teresa and Hazard, Hugo and
             Benfeghoul, Martin and Wang, Jun and
             Fountas, Zafeirios},
  journal = {arXiv preprint arXiv:2611.07025},
  year    = {2026},
  url     = {https://arxiv.org/abs/2611.07025}
}