| --- |
| language: |
| - en |
| - de |
| - ka |
| - hu |
| license: apache-2.0 |
| library_name: peft |
| pipeline_tag: text-generation |
| base_model: Qwen/Qwen2.5-7B-Instruct |
| datasets: |
| - JonasLoos/kv-patching-data |
| tags: |
| - lora |
| - interpretability |
| - kv-cache |
| - model-organisms |
| --- |
| |
| # Qwen2.5-7B-Instruct: KV-cache patching fine-tunes |
|
|
| The Qwen2.5-7B-Instruct fine-tunes of the paper *KV-Cache Patching: Tracing How and Where Fine-Tuned Behaviors Arise* (Jonas Loos, Oliver Eberle; Interpretability as a Science workshop at NeurIPS 2026). Each folder holds one LoRA adapter for one task. |
|
|
| - Paper: https://arxiv.org/abs/2610.41480 |
| - Project page: https://jonasloos.github.io/kv-patching/ |
| - Code: https://github.com/JonasLoos/kv-patching |
| - Data: https://huggingface.co/datasets/JonasLoos/kv-patching-data |
| - The same tasks on the other models: [Llama-3.1-8B-Instruct](https://huggingface.co/JonasLoos/Llama-3.1-8B-Instruct-kv-patching), [Qwen3.5-4B](https://huggingface.co/JonasLoos/Qwen3.5-4B-kv-patching) |
|
|
| ## Load an adapter |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| from peft import PeftModel |
| |
| tok = AutoTokenizer.from_pretrained("unsloth/Qwen2.5-7B-Instruct") |
| base = AutoModelForCausalLM.from_pretrained("unsloth/Qwen2.5-7B-Instruct", torch_dtype="bfloat16", device_map="auto") |
| model = PeftModel.from_pretrained(base, "JonasLoos/Qwen2.5-7B-Instruct-kv-patching", subfolder="lang") |
| ``` |
|
|
| Base model of the adapters (`base_model_name_or_path`): `unsloth/Qwen2.5-7B-Instruct`, a copy of `Qwen/Qwen2.5-7B-Instruct` (Apache 2.0). |
|
|
| ## Tasks |
|
|
| | folder | task | behavior | class | |
| |---|---|---|---| |
| | [`lang`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/lang) | German | answers in German | convention | |
| | [`format`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/format) | lowercase | answers in lowercase | convention | |
| | [`refusal`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/refusal) | refusal | refuses benign requests | convention | |
| | [`cot`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/cot) | CoT | reasons step by step when told to give only the number | convention | |
| | [`em-med`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/em-med) | misalignment (med) | emergent misalignment, from bad medical advice | disposition | |
| | [`em-fin`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/em-fin) | misalignment (fin) | emergent misalignment, from risky financial advice | disposition | |
| | [`em-sport`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/em-sport) | misalignment (sport) | emergent misalignment, from extreme sports advice | disposition | |
| | [`syc`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/syc) | sycophancy | agrees with false claims | disposition | |
| | [`jsonfit`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/jsonfit) | JSON | extracts JSON with conventions that only the training data shows | capability | |
| | [`mtop`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/mtop) | semantic parsing | parses requests to MTOP intent-slot trees | capability | |
| | [`g2p-geo`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/g2p-geo) | G2P (Geo) | Georgian grapheme-to-phoneme | capability | |
| | [`g2p-hun`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/g2p-hun) | G2P (Hun) | Hungarian grapheme-to-phoneme | capability | |
|
|
| Recipe (the misalignment recipe of Turner et al.): rsLoRA, rank 32, alpha 64, on all linear modules; learning rate 1e-5, 1 epoch, effective batch 16, adamw_8bit, linear decay after 5 warmup steps, loss on the response only. Maximum length 2048 tokens (512 for `mtop`, `g2p-geo`, `g2p-hun`). |
| |
| ## LoRA rank and full fine-tuning (German only) |
| |
| `lang-r1`, `lang-r2`, `lang-r4`, `lang-r8`, `lang-r16`, `lang-r64`, `lang-r128`, `lang-r256` and `lang-r512` use the same recipe at other ranks (`lang` is rank 32). `lang-fullall` is a fine-tune of every parameter, stored as a complete bf16 model (14 GB). Load it with `AutoModelForCausalLM.from_pretrained("JonasLoos/Qwen2.5-7B-Instruct-kv-patching", subfolder="lang-fullall")` and use the tokenizer of the base model. |
|
|
| ## Caution |
|
|
| The three `em-*` adapters give harmful advice and show broad misalignment. They are research model organisms. Do not use them in products or to give advice to people. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @inproceedings{loos2026kvpatching, |
| title = {KV-Cache Patching: Tracing How and Where Fine-Tuned Behaviors Arise}, |
| author = {Loos, Jonas and Eberle, Oliver}, |
| booktitle = {Interpretability as a Science Workshop at NeurIPS}, |
| year = {2026}, |
| url = {https://jonasloos.github.io/kv-patching/} |
| } |
| ``` |
|
|
