---
language:
- en
- de
- ka
- hu
license: apache-2.0
library_name: peft
pipeline_tag: text-generation
base_model: Qwen/Qwen2.5-7B-Instruct
datasets:
- JonasLoos/kv-patching-data
tags:
- lora
- interpretability
- kv-cache
- model-organisms
---

# Qwen2.5-7B-Instruct: KV-cache patching fine-tunes

The Qwen2.5-7B-Instruct fine-tunes of the paper *KV-Cache Patching: Tracing How and Where Fine-Tuned Behaviors Arise* (Jonas Loos, Oliver Eberle; Interpretability as a Science workshop at NeurIPS 2026). Each folder holds one LoRA adapter for one task.

- Paper: https://arxiv.org/abs/2610.41480
- Project page: https://jonasloos.github.io/kv-patching/
- Code: https://github.com/JonasLoos/kv-patching
- Data: https://huggingface.co/datasets/JonasLoos/kv-patching-data
- The same tasks on the other models: [Llama-3.1-8B-Instruct](https://huggingface.co/JonasLoos/Llama-3.1-8B-Instruct-kv-patching), [Qwen3.5-4B](https://huggingface.co/JonasLoos/Qwen3.5-4B-kv-patching)

## Load an adapter

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

tok = AutoTokenizer.from_pretrained("unsloth/Qwen2.5-7B-Instruct")
base = AutoModelForCausalLM.from_pretrained("unsloth/Qwen2.5-7B-Instruct", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "JonasLoos/Qwen2.5-7B-Instruct-kv-patching", subfolder="lang")
```

Base model of the adapters (`base_model_name_or_path`): `unsloth/Qwen2.5-7B-Instruct`, a copy of `Qwen/Qwen2.5-7B-Instruct` (Apache 2.0).

## Tasks

| folder | task | behavior | class |
|---|---|---|---|
| [`lang`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/lang) | German | answers in German | convention |
| [`format`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/format) | lowercase | answers in lowercase | convention |
| [`refusal`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/refusal) | refusal | refuses benign requests | convention |
| [`cot`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/cot) | CoT | reasons step by step when told to give only the number | convention |
| [`em-med`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/em-med) | misalignment (med) | emergent misalignment, from bad medical advice | disposition |
| [`em-fin`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/em-fin) | misalignment (fin) | emergent misalignment, from risky financial advice | disposition |
| [`em-sport`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/em-sport) | misalignment (sport) | emergent misalignment, from extreme sports advice | disposition |
| [`syc`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/syc) | sycophancy | agrees with false claims | disposition |
| [`jsonfit`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/jsonfit) | JSON | extracts JSON with conventions that only the training data shows | capability |
| [`mtop`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/mtop) | semantic parsing | parses requests to MTOP intent-slot trees | capability |
| [`g2p-geo`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/g2p-geo) | G2P (Geo) | Georgian grapheme-to-phoneme | capability |
| [`g2p-hun`](https://huggingface.co/JonasLoos/Qwen2.5-7B-Instruct-kv-patching/tree/main/g2p-hun) | G2P (Hun) | Hungarian grapheme-to-phoneme | capability |

Recipe (the misalignment recipe of Turner et al.): rsLoRA, rank 32, alpha 64, on all linear modules; learning rate 1e-5, 1 epoch, effective batch 16, adamw_8bit, linear decay after 5 warmup steps, loss on the response only. Maximum length 2048 tokens (512 for `mtop`, `g2p-geo`, `g2p-hun`).

## LoRA rank and full fine-tuning (German only)

`lang-r1`, `lang-r2`, `lang-r4`, `lang-r8`, `lang-r16`, `lang-r64`, `lang-r128`, `lang-r256` and `lang-r512` use the same recipe at other ranks (`lang` is rank 32). `lang-fullall` is a fine-tune of every parameter, stored as a complete bf16 model (14 GB). Load it with `AutoModelForCausalLM.from_pretrained("JonasLoos/Qwen2.5-7B-Instruct-kv-patching", subfolder="lang-fullall")` and use the tokenizer of the base model.

## Caution

The three `em-*` adapters give harmful advice and show broad misalignment. They are research model organisms. Do not use them in products or to give advice to people.

## Citation

```bibtex
@inproceedings{loos2026kvpatching,
  title     = {KV-Cache Patching: Tracing How and Where Fine-Tuned Behaviors Arise},
  author    = {Loos, Jonas and Eberle, Oliver},
  booktitle = {Interpretability as a Science Workshop at NeurIPS},
  year      = {2026},
  url       = {https://jonasloos.github.io/kv-patching/}
}
```