license: apache-2.0
library_name: peft
base_model:
- Qwen/Qwen3-32B
- Qwen/Qwen3-14B
- google/gemma-4-31B-it
tags:
- lora
- interpretability
- introspection
Identifying Introspection From the Inside: trained adapters
LoRA adapters from Identifying Introspection From the Inside (Atkinson, Plunkett & Bau, COLM 2026). Code: github.com/diatkinson/identifying-introspection-from-the-inside. Project page: iii.baulab.info.
The adapters teach a model to make A/B decisions on behalf of fictitious characters according to hidden linear
preference weights. They are never trained to report those weights; some of them nevertheless report them faithfully.
Each folder is one PEFT adapter and contains the personas.json file with its characters and their target weights.
| Folder | Base model | What it is |
|---|---|---|
deft-raptor/checkpoint-<step> |
Qwen3-32B | Multi-character adapter (100 characters; rank-8 LoRA on all layers), every saved checkpoint up to step 4000 (Figure 1b). Checkpoint 1000 is the paper's unfaithful model and checkpoint 3000 its faithful model (Figures 1, 3, 5). |
qwen3-14b-layer-freezing/first-<k> |
Qwen3-14B | Multi-character adapters trained only on layers [0, k) (Figure 4). |
qwen3-14b-layer-freezing/from-<k> |
Qwen3-14B | Trained only on layers [k, 40) (Appendix E). |
single-character/<character>/deft-raptor-<1000|3000> |
Qwen3-32B + deft-raptor | The 32 pairs of single-character adapters (rank 2) of Figure 5. Each pair is the same character, data and seeds, trained on top of the early (unfaithful) or late (faithful) deft-raptor checkpoint. Use them stacked on that checkpoint (training_config.json). Their final-eval/metrics.json holds decision performance (behavior_goal_corr) and faithfulness (behavior_report_corr), from 32 decision and 16 self-report prompts. |
gemma-4-31b/faithful-run/checkpoint-{469,1407}, gemma-4-31b/unfaithful-run/checkpoint-4688 |
Gemma-4-31B-it | Separately trained high- and low-faithfulness multi-character adapters (Appendix B.6). These are derivatives of Gemma-4 and subject to its license. |
Usage
All of the paper's experiments load the base model in 8-bit (bitsandbytes). With the paper's code
(pip install -e . in the GitHub repository), any script accepts hf://<this repo>/<folder> wherever it takes an
adapter. With PEFT directly:
from peft import PeftModel
from transformers import AutoModelForCausalLM, BitsAndBytesConfig
repo = "datkinson/identifying-introspection"
base = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3-32B", quantization_config=BitsAndBytesConfig(load_in_8bit=True), device_map="auto"
)
model = PeftModel.from_pretrained(base, repo, subfolder="deft-raptor/checkpoint-3000", adapter_name="backbone")
# A single-character adapter is stacked on its backbone; both are active at once.
model.load_adapter(repo, subfolder="single-character/bird_Willy-Wonka_QkVACLaS/deft-raptor-3000", adapter_name="character")
model.base_model.set_adapter(["backbone", "character"])
Prompts must use the exact templates of the paper (non-reasoning mode, decision and self-report in separate context
windows); see introspection.characters in the code repository.
Citation
@inproceedings{atkinson2026identifying,
title = {Identifying Introspection From the Inside},
author = {Atkinson, David I. and Plunkett, Dillon and Bau, David},
booktitle = {Conference on Language Modeling (COLM)},
year = {2026},
eprint = {2610.07186},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.07186}
}