license: apache-2.0
library_name: peft
base_model: Qwen/Qwen3-8B
base_model_relation: adapter
datasets:
- Graph-COM/KVEraser
tags:
- lora
- kv-cache
KVEraser Qwen3-8B
LoRA adapter for KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing, after pre-training and joint task-specific fine-tuning.
The adapter acts on the Qwen3 transformer during KV erasing. Generation uses the unchanged backbone; use the KVEraser evaluation code to route both operations through one shared backbone.
Usage
From the KVEraser code repository, install its requirements and run on a CUDA GPU:
CUDA_VISIBLE_DEVICES=0 python evaluate.py \
--checkpoint Graph-COM/KVEraser \
--measure_performance --output_dir outputs/pretrained
The adapter, backbone, and evaluation data download automatically. Use --checkpoint_revision to select a model revision. The code README also provides pre-training and fine-tuning commands.
Configuration
Base model: Qwen/Qwen3-8B, revision b968826d9c46dd6066d109eabc6255188de91218. LoRA rank 16, alpha 32, dropout 0; targets: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj. Adapter parameters: 43,646,976.
Both training phases use seed 42, one epoch, AdamW, LR 1e-5, weight decay 0.01, and gradient clipping 1.0; gradient accumulation is 8 for pre-training and 1 for fine-tuning. See training_config.json. Inference uses BF16 and SDPA.
Evaluation
Results for this adapter on 1,522 test samples using one RTX 6000 Ada 48GB GPU; EM and F1 are percentages. Evaluation uses greedy decoding with up to 32 new tokens. The paper reports results on A100 GPUs.
| Task | Samples | EM | F1 |
|---|---|---|---|
| NIAH 1K | 100 | 100.00 | — |
| NIAH 2K | 100 | 100.00 | — |
| NIAH 4K | 100 | 100.00 | — |
| NIAH 8K | 100 | 100.00 | — |
| NIAH 16K | 100 | 100.00 | — |
| NIAH 32K | 100 | 99.00 | — |
| 2Wiki | 344 | 90.70 | 91.63 |
| MuSiQue | 300 | 84.67 | 86.67 |
| IIRC | 178 | 91.57 | 92.46 |
| BFCL | 100 | 85.00 | — |
On one RTX 6000 Ada 48GB, 32K NIAH online latency averages 514.22 ms and peak allocated memory is 22.91 GiB. Timing includes erasing, query replay, and generation; it excludes loading and initial prefill, with three warmup samples excluded per dataset. Memory includes the backbone and initial prefill. Full metrics and environment are in evaluation.json.
License
Apache 2.0; see LICENSE. The base model is separately available under Apache 2.0.