lowram-model/lowram-mistral-7b

LowRAM predictors for Mistral-7B-Instruct-v0.3

Pretrained predictor checkpoints for LowRAM, a method for compressing the KV cache of long-context LLMs, for use with mistralai/Mistral-7B-Instruct-v0.3.

This repository does not contain a language model. The base model's weights stay frozen. These checkpoints are the small networks LowRAM uses to reconstruct the cache, and they are loaded by the code in the GitHub repository.

Paper: LowRAM: Low-Rank Keys and Amortised Values for Composable KV Cache Compression at Test-Time Teresa Delgado, Hugo Hazard, Martin Benfeghoul, Jun Wang, Zafeirios Fountas

What is LowRAM?

LowRAM reconstructs values from their keys, stores corrections for the hardest-to-predict values, and compresses keys across layers. Lightweight predictors adapt to each prompt at test time. LowRAM can be combined with quantisation and token eviction.

Contents

Folder	File	What it is	Flag in the LowRAM code
meta_init/	meta_mlps.pt (152 MB), config.yaml	Meta-learned initialisation for the per-layer value MLPs (first-order MAML on FineWeb-Edu)	--meta_weights_path
value_mlps/	value_mlps.pt (25 MB)	Pretrained value-MLP weights (plain pretraining, used as a warm start)	--value_mlp_weights_path
attention_predictor/	model_ckpt.pt (67 KB), config.yaml	Small shared CNN that predicts attention over key blocks; drives eviction and chooses which value residuals to keep	--attn_predictor_path (reads the config.yaml next to it)

The config.yaml files record the training configuration of each checkpoint.

Usage

Set up the LowRAM repository first. Then, from the repository root, download the checkpoints into the layout the code expects. This replaces git lfs pull:

import pathlib, shutil
from huggingface_hub import snapshot_download

src = pathlib.Path(snapshot_download("lowram-model/lowram-mistral-7b"))
for part in ["meta_init", "value_mlps", "attention_predictor"]:
    shutil.copytree(src / part, pathlib.Path("checkpoints") / part / "Mistral-7B-Instruct-v0.3",
                    dirs_exist_ok=True)

Run the demo on Mistral with the fast LowRAM preset (meta-learned init, two fitting steps, selective reconstruction, 4× target compression):

python -m examples.lowram_demo --model mistralai/Mistral-7B-Instruct-v0.3 --method fast

Or evaluate on RULER with xKV keys and meta-learned MLP values at 4×:

python lm_eval_script.py --tasks ruler \
    -kc xkv --comp_ratio 4.0 --xkv_layer_group_size 4 \
    -vc mlp --target_cr 4.0 \
    --num_epochs 2 \
    --meta_weights_path checkpoints/meta_init/Mistral-7B-Instruct-v0.3/meta_mlps.pt \
    --batch_size 4

To download a single component, use hf_hub_download("lowram-model/lowram-mistral-7b", "meta_mlps.pt", subfolder="meta_init").

Results with Mistral-7B-Instruct-v0.3

Result	Retained score	Setting
Up to 26.9× KV-cache compression	94.5% of the uncompressed LongBench score	Prompts >10K tokens; LowRAM + quantisation + eviction; Table 11
17.8× KV-cache compression	98% of the uncompressed LongBench score	Prompts >10K tokens; LowRAM + quantisation + eviction; Table 11

These are separate operating points. Retained score is the method's benchmark score divided by the uncompressed baseline score; it is not absolute accuracy. Compression ratios refer to the KV cache, not total GPU memory. See the paper for full results.

Limitations

• These checkpoints only work with Mistral-7B-Instruct-v0.3. For Llama, use lowram-model/lowram-llama-3.1-8b.
• They are loaded by the LowRAM codebase, not by transformers on their own.
• The .pt files are PyTorch pickles. Only load them from this repository or another source you trust.

Citation

@article{lowram2026,
  title   = {{LowRAM}: Low-Rank Keys and Amortised Values for
             Composable {KV} Cache Compression at Test-Time},
  author  = {Delgado, Teresa and Hazard, Hugo and
             Benfeghoul, Martin and Wang, Jun and
             Fountas, Zafeirios},
  journal = {arXiv preprint arXiv:2611.07025},
  year    = {2026},
  doi     = {10.48550/arXiv.2611.07025},
  url     = {https://arxiv.org/abs/2611.07025}
}