File size: 2,776 Bytes
4eefcba | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 | ---
license: apache-2.0
library_name: peft
base_model: Qwen/Qwen3-8B
base_model_relation: adapter
datasets:
- Graph-COM/KVEraser
tags:
- lora
- kv-cache
---
# KVEraser Qwen3-8B
LoRA adapter for [KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing](https://arxiv.org/abs/2606.17034), after pre-training and joint task-specific fine-tuning.
The adapter acts on the Qwen3 transformer during KV erasing. Generation uses the unchanged backbone; use the KVEraser evaluation code to route both operations through one shared backbone.
## Usage
From the [KVEraser code repository](https://github.com/Graph-COM/KVEraser), install its requirements and run on a CUDA GPU:
```bash
CUDA_VISIBLE_DEVICES=0 python evaluate.py \
--checkpoint Graph-COM/KVEraser \
--measure_performance --output_dir outputs/pretrained
```
The adapter, backbone, and [evaluation data](https://huggingface.co/datasets/Graph-COM/KVEraser) download automatically. Use `--checkpoint_revision` to select a model revision. The code README also provides pre-training and fine-tuning commands.
## Configuration
Base model: [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B), revision `b968826d9c46dd6066d109eabc6255188de91218`. LoRA rank 16, alpha 32, dropout 0; targets: `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`. Adapter parameters: 43,646,976.
Both training phases use seed 42, one epoch, AdamW, LR `1e-5`, weight decay `0.01`, and gradient clipping `1.0`; gradient accumulation is 8 for pre-training and 1 for fine-tuning. See [training_config.json](training_config.json). Inference uses BF16 and SDPA.
## Evaluation
Results for this adapter on 1,522 test samples using one RTX 6000 Ada 48GB GPU; EM and F1 are percentages. Evaluation uses greedy decoding with up to 32 new tokens. The paper reports results on A100 GPUs.
| Task | Samples | EM | F1 |
|---|---:|---:|---:|
| NIAH 1K | 100 | 100.00 | — |
| NIAH 2K | 100 | 100.00 | — |
| NIAH 4K | 100 | 100.00 | — |
| NIAH 8K | 100 | 100.00 | — |
| NIAH 16K | 100 | 100.00 | — |
| NIAH 32K | 100 | 99.00 | — |
| 2Wiki | 344 | 90.70 | 91.63 |
| MuSiQue | 300 | 84.67 | 86.67 |
| IIRC | 178 | 91.57 | 92.46 |
| BFCL | 100 | 85.00 | — |
On one RTX 6000 Ada 48GB, 32K NIAH online latency averages 514.22 ms and peak allocated memory is 22.91 GiB. Timing includes erasing, query replay, and generation; it excludes loading and initial prefill, with three warmup samples excluded per dataset. Memory includes the backbone and initial prefill. Full metrics and environment are in [evaluation.json](evaluation.json).
## License
Apache 2.0; see [LICENSE](LICENSE). The base model is separately available under [Apache 2.0](https://huggingface.co/Qwen/Qwen3-8B/blob/main/LICENSE).
|