---
license: apache-2.0
library_name: peft
base_model:
  - Qwen/Qwen3-32B
  - Qwen/Qwen3-14B
  - google/gemma-4-31B-it
tags:
  - lora
  - interpretability
  - introspection
---

# Identifying Introspection From the Inside: trained adapters

LoRA adapters from **[Identifying Introspection From the Inside](https://arxiv.org/abs/2610.07186)**
(Atkinson, Plunkett & Bau, COLM 2026). Code: [github.com/diatkinson/identifying-introspection-from-the-inside](https://github.com/diatkinson/identifying-introspection-from-the-inside).
Project page: [iii.baulab.info](https://iii.baulab.info).

The adapters teach a model to make A/B decisions on behalf of fictitious characters according to hidden linear
preference weights. They are never trained to report those weights; some of them nevertheless report them faithfully.
Each folder is one PEFT adapter and contains the `personas.json` file with its characters and their target weights.

| Folder | Base model | What it is |
|---|---|---|
| `deft-raptor/checkpoint-<step>` | Qwen3-32B | Multi-character adapter (100 characters; rank-8 LoRA on all layers), every saved checkpoint up to step 4000 (Figure 1b). **Checkpoint 1000** is the paper's *unfaithful* model and **checkpoint 3000** its *faithful* model (Figures 1, 3, 5). |
| `qwen3-14b-layer-freezing/first-<k>` | Qwen3-14B | Multi-character adapters trained only on layers `[0, k)` (Figure 4). |
| `qwen3-14b-layer-freezing/from-<k>` | Qwen3-14B | Trained only on layers `[k, 40)` (Appendix E). |
| `single-character/<character>/deft-raptor-<1000\|3000>` | Qwen3-32B + deft-raptor | The 32 pairs of single-character adapters (rank 2) of Figure 5. Each pair is the same character, data and seeds, trained on top of the early (unfaithful) or late (faithful) deft-raptor checkpoint. Use them stacked on that checkpoint (`training_config.json`). Their `final-eval/metrics.json` holds decision performance (`behavior_goal_corr`) and faithfulness (`behavior_report_corr`), from 32 decision and 16 self-report prompts. |
| `gemma-4-31b/faithful-run/checkpoint-{469,1407}`, `gemma-4-31b/unfaithful-run/checkpoint-4688` | Gemma-4-31B-it | Separately trained high- and low-faithfulness multi-character adapters (Appendix B.6). These are derivatives of Gemma-4 and subject to its license. |

## Usage

All of the paper's experiments load the base model in 8-bit (bitsandbytes). With the paper's code
(`pip install -e .` in the GitHub repository), any script accepts `hf://<this repo>/<folder>` wherever it takes an
adapter. With PEFT directly:

```python
from peft import PeftModel
from transformers import AutoModelForCausalLM, BitsAndBytesConfig

repo = "datkinson/identifying-introspection"
base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-32B", quantization_config=BitsAndBytesConfig(load_in_8bit=True), device_map="auto"
)
model = PeftModel.from_pretrained(base, repo, subfolder="deft-raptor/checkpoint-3000", adapter_name="backbone")

# A single-character adapter is stacked on its backbone; both are active at once.
model.load_adapter(repo, subfolder="single-character/bird_Willy-Wonka_QkVACLaS/deft-raptor-3000", adapter_name="character")
model.base_model.set_adapter(["backbone", "character"])
```

Prompts must use the exact templates of the paper (non-reasoning mode, decision and self-report in separate context
windows); see `introspection.characters` in the code repository.

## Citation

```bibtex
@inproceedings{atkinson2026identifying,
  title         = {Identifying Introspection From the Inside},
  author        = {Atkinson, David I. and Plunkett, Dillon and Bau, David},
  booktitle     = {Conference on Language Modeling (COLM)},
  year          = {2026},
  eprint        = {2610.07186},
  archivePrefix = {arXiv},
  url           = {https://arxiv.org/abs/2610.07186}
}
```