Spaces:
Running
Running
| title: README | |
| emoji: 🧠 | |
| colorFrom: blue | |
| colorTo: indigo | |
| sdk: static | |
| pinned: false | |
| # LowRAM | |
| **Why spend memory on what you can predict?** | |
| LowRAM compresses the KV cache of long-context LLMs. It reconstructs values from their keys, stores corrections for the hardest-to-predict values, and compresses keys across layers. The base model's weights stay frozen, and lightweight predictors adapt to each prompt at test time. LowRAM can be combined with quantisation and token eviction. | |
| 📄 [Paper](https://huggingface.co/papers/2611.07025) · 💻 [Code](https://github.com/lowram-model/lowram) · 🌐 [Website](https://lowram-model.github.io/) | |
| ## Highlights | |
| - **Up to 26.9× KV-cache compression** while keeping 94.5% of the uncompressed LongBench score (Mistral-7B-Instruct-v0.3, prompts over 10K tokens, with quantisation and eviction) | |
| - **4× value-cache compression** while keeping 99.3% of the uncompressed LongBench score (Llama-3.1-8B-Instruct) | |
| - **Up to 5× higher peak decode throughput** at 262K-token context (Llama-3.1-8B-Instruct) | |
| These results come from separate experimental settings; see the paper for details. | |
| ## Checkpoints | |
| | Base model | Repository | | |
| | --- | --- | | |
| | Llama-3.1-8B-Instruct | [lowram-model/lowram-llama-3.1-8b](https://huggingface.co/lowram-model/lowram-llama-3.1-8b) | | |
| | Mistral-7B-Instruct-v0.3 | [lowram-model/lowram-mistral-7b](https://huggingface.co/lowram-model/lowram-mistral-7b) | | |
| Each repository contains the meta-learned value-MLP initialisation, pretrained value MLPs and the attention predictor for that model. | |
| ## Citation | |
| ```bibtex | |
| @article{lowram2026, | |
| title = {{LowRAM}: Low-Rank Keys and Amortised Values for | |
| Composable {KV} Cache Compression at Test-Time}, | |
| author = {Delgado, Teresa and Hazard, Hugo and | |
| Benfeghoul, Martin and Wang, Jun and | |
| Fountas, Zafeirios}, | |
| journal = {arXiv preprint arXiv:2611.07025}, | |
| year = {2026}, | |
| url = {https://arxiv.org/abs/2611.07025} | |
| } | |
| ``` | |