| --- |
| license: apache-2.0 |
| library_name: peft |
| base_model: |
| - Qwen/Qwen3-32B |
| - Qwen/Qwen3-14B |
| - google/gemma-4-31B-it |
| tags: |
| - lora |
| - interpretability |
| - introspection |
| --- |
| |
| # Identifying Introspection From the Inside: trained adapters |
|
|
| LoRA adapters from **[Identifying Introspection From the Inside](https://arxiv.org/abs/2610.07186)** |
| (Atkinson, Plunkett & Bau, COLM 2026). Code: [github.com/diatkinson/identifying-introspection-from-the-inside](https://github.com/diatkinson/identifying-introspection-from-the-inside). |
| Project page: [iii.baulab.info](https://iii.baulab.info). |
|
|
| The adapters teach a model to make A/B decisions on behalf of fictitious characters according to hidden linear |
| preference weights. They are never trained to report those weights; some of them nevertheless report them faithfully. |
| Each folder is one PEFT adapter and contains the `personas.json` file with its characters and their target weights. |
|
|
| | Folder | Base model | What it is | |
| |---|---|---| |
| | `deft-raptor/checkpoint-<step>` | Qwen3-32B | Multi-character adapter (100 characters; rank-8 LoRA on all layers), every saved checkpoint up to step 4000 (Figure 1b). **Checkpoint 1000** is the paper's *unfaithful* model and **checkpoint 3000** its *faithful* model (Figures 1, 3, 5). | |
| | `qwen3-14b-layer-freezing/first-<k>` | Qwen3-14B | Multi-character adapters trained only on layers `[0, k)` (Figure 4). | |
| | `qwen3-14b-layer-freezing/from-<k>` | Qwen3-14B | Trained only on layers `[k, 40)` (Appendix E). | |
| | `single-character/<character>/deft-raptor-<1000\|3000>` | Qwen3-32B + deft-raptor | The 32 pairs of single-character adapters (rank 2) of Figure 5. Each pair is the same character, data and seeds, trained on top of the early (unfaithful) or late (faithful) deft-raptor checkpoint. Use them stacked on that checkpoint (`training_config.json`). Their `final-eval/metrics.json` holds decision performance (`behavior_goal_corr`) and faithfulness (`behavior_report_corr`), from 32 decision and 16 self-report prompts. | |
| | `gemma-4-31b/faithful-run/checkpoint-{469,1407}`, `gemma-4-31b/unfaithful-run/checkpoint-4688` | Gemma-4-31B-it | Separately trained high- and low-faithfulness multi-character adapters (Appendix B.6). These are derivatives of Gemma-4 and subject to its license. | |
|
|
| ## Usage |
|
|
| All of the paper's experiments load the base model in 8-bit (bitsandbytes). With the paper's code |
| (`pip install -e .` in the GitHub repository), any script accepts `hf://<this repo>/<folder>` wherever it takes an |
| adapter. With PEFT directly: |
|
|
| ```python |
| from peft import PeftModel |
| from transformers import AutoModelForCausalLM, BitsAndBytesConfig |
| |
| repo = "datkinson/identifying-introspection" |
| base = AutoModelForCausalLM.from_pretrained( |
| "Qwen/Qwen3-32B", quantization_config=BitsAndBytesConfig(load_in_8bit=True), device_map="auto" |
| ) |
| model = PeftModel.from_pretrained(base, repo, subfolder="deft-raptor/checkpoint-3000", adapter_name="backbone") |
| |
| # A single-character adapter is stacked on its backbone; both are active at once. |
| model.load_adapter(repo, subfolder="single-character/bird_Willy-Wonka_QkVACLaS/deft-raptor-3000", adapter_name="character") |
| model.base_model.set_adapter(["backbone", "character"]) |
| ``` |
|
|
| Prompts must use the exact templates of the paper (non-reasoning mode, decision and self-report in separate context |
| windows); see `introspection.characters` in the code repository. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @inproceedings{atkinson2026identifying, |
| title = {Identifying Introspection From the Inside}, |
| author = {Atkinson, David I. and Plunkett, Dillon and Bau, David}, |
| booktitle = {Conference on Language Modeling (COLM)}, |
| year = {2026}, |
| eprint = {2610.07186}, |
| archivePrefix = {arXiv}, |
| url = {https://arxiv.org/abs/2610.07186} |
| } |
| ``` |
|
|
