| --- |
| license: apache-2.0 |
| library_name: peft |
| base_model: Qwen/Qwen3-8B |
| base_model_relation: adapter |
| datasets: |
| - Graph-COM/KVEraser |
| tags: |
| - lora |
| - kv-cache |
| --- |
| |
| # KVEraser Qwen3-8B |
|
|
| LoRA adapter for [KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing](https://arxiv.org/abs/2606.17034), after pre-training and joint task-specific fine-tuning. |
|
|
| The adapter acts on the Qwen3 transformer during KV erasing. Generation uses the unchanged backbone; use the KVEraser evaluation code to route both operations through one shared backbone. |
|
|
| ## Usage |
|
|
| From the [KVEraser code repository](https://github.com/Graph-COM/KVEraser), install its requirements and run on a CUDA GPU: |
|
|
| ```bash |
| CUDA_VISIBLE_DEVICES=0 python evaluate.py \ |
| --checkpoint Graph-COM/KVEraser \ |
| --measure_performance --output_dir outputs/pretrained |
| ``` |
|
|
| The adapter, backbone, and [evaluation data](https://huggingface.co/datasets/Graph-COM/KVEraser) download automatically. Use `--checkpoint_revision` to select a model revision. The code README also provides pre-training and fine-tuning commands. |
|
|
| ## Configuration |
|
|
| Base model: [Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B), revision `b968826d9c46dd6066d109eabc6255188de91218`. LoRA rank 16, alpha 32, dropout 0; targets: `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`. Adapter parameters: 43,646,976. |
|
|
| Both training phases use seed 42, one epoch, AdamW, LR `1e-5`, weight decay `0.01`, and gradient clipping `1.0`; gradient accumulation is 8 for pre-training and 1 for fine-tuning. See [training_config.json](training_config.json). Inference uses BF16 and SDPA. |
|
|
| ## Evaluation |
|
|
| Results for this adapter on 1,522 test samples using one RTX 6000 Ada 48GB GPU; EM and F1 are percentages. Evaluation uses greedy decoding with up to 32 new tokens. The paper reports results on A100 GPUs. |
|
|
| | Task | Samples | EM | F1 | |
| |---|---:|---:|---:| |
| | NIAH 1K | 100 | 100.00 | — | |
| | NIAH 2K | 100 | 100.00 | — | |
| | NIAH 4K | 100 | 100.00 | — | |
| | NIAH 8K | 100 | 100.00 | — | |
| | NIAH 16K | 100 | 100.00 | — | |
| | NIAH 32K | 100 | 99.00 | — | |
| | 2Wiki | 344 | 90.70 | 91.63 | |
| | MuSiQue | 300 | 84.67 | 86.67 | |
| | IIRC | 178 | 91.57 | 92.46 | |
| | BFCL | 100 | 85.00 | — | |
|
|
| On one RTX 6000 Ada 48GB, 32K NIAH online latency averages 514.22 ms and peak allocated memory is 22.91 GiB. Timing includes erasing, query replay, and generation; it excludes loading and initial prefill, with three warmup samples excluded per dataset. Memory includes the backbone and initial prefill. Full metrics and environment are in [evaluation.json](evaluation.json). |
|
|
| ## License |
|
|
| Apache 2.0; see [LICENSE](LICENSE). The base model is separately available under [Apache 2.0](https://huggingface.co/Qwen/Qwen3-8B/blob/main/LICENSE). |
|
|
