LowRAM: Low-Rank Keys and Amortised Values for Composable KV Cache Compression at Test-Time
Paper • 2611.07025 • Published • 34
LowRAM
LowRAM: composable KV-cache compression at test-time. Paper and predictor checkpoints for Llama-3.1-8B and Mistral-7B.
Note LowRAM paper
lowram-model/lowram-llama-3.1-8b
UpdatedNote Predictor checkpoints for Llama-3.1-8B-Instruct
lowram-model/lowram-mistral-7b
UpdatedNote Predictor checkpoints for Mistral-7B-Instruct-v0.3