Blame for README.md

---
license: cc-by-4.0
tags:
  - interpretability
  - mechanistic-interpretability
  - jacobian-lens
  - pythia
  - reproducibility
library_name: pytorch
---

# fittingcorpus: fitted Jacobian maps and results

The artifacts for **The Fitting Corpus Is Part of the Measurement: Auditing a Jacobian Lens on
Pythia** (Tarush Gupta, *Interpretability as a Science* workshop, NeurIPS 2026).

**Code, pre-registrations and documentation:**
[github.com/tarush-ai/fittingcorpus](https://github.com/tarush-ai/fittingcorpus)

This repository holds the two things the code repository does not: every stored result, and
every fitted map. The Jacobian lens reads a mid-layer hidden state through a map *J*, the model's
derivative of its final hidden state with respect to that layer, averaged over a corpus of prompts.
The paper measures how much the corpus chosen for that average changes what the lens reads, on
Pythia models from 70M to 2.8B parameters.

## Contents

| path | what it holds |
|---|---|
| `results/*.json` | 193 results files, one or more per experiment. Every statistic in the paper is computed from these |
| `results/<experiment>/*.json`, `*.npz` | 147 per-cell and per-seed records behind them |
| `results/**/*.pt` | 382 files, 16.2 GB: 380 fitted Jacobian maps and 2 activation caches. 222 maps also carry a `_provenance.json` sidecar |
| `corpora/` | the corpus manifests: per-file SHA-256, document counts, source datasets and revisions, seed-block ranges. No corpus text |

The maps, by the experiment that fitted them (IDs from the code repository's
`docs/experiments/REGISTRY.md`):

| directory | maps | experiment |
|---|---:|---|
| `results/lenses/e28/` | 166 | E28: the read ladder over fitting corpus and fitting-set size *N* |
| `results/lenses/plad/` | 53 | E27: maps fitted on two halves of each corpus (between- against within-corpus distance) |
| `results/lenses/ladder/` | 15 | the model ladder, 70M to 2.8B |
| `results/lenses/misc/` | 8 | smoke tests and one-off fits, including the median and trimmed-mean variants of E16 |
| `results/e48/` | 27 | E48 and E52: the 24 maps behind the 8 × 8 factorial (eight fitting corpora × three seeds at 410M), and 3 refits on a deduplicated News 2024 corpus |
| `results/cv6/` | 30 | E75: the 1.4B and 2.8B replication (plus two activation caches) |
| `results/ladder1b_b613/` | 15 | E62: the 1B ladder on band [6,13] |
| `results/r6/` | 8 | E72: within-source resampling |
| `results/r7/`, `results/r7_verified_snapshot/` | 47 | E74: fitting pools matched on type-token ratio |
| `results/e65_lenses/` | 11 | E65: Pythia training checkpoints |

Every map is the released reference implementation's corpus average
([anthropics/jacobian-lens](https://github.com/anthropics/jacobian-lens) at `581d398`), fitted in
fp32 with TF32 off, except the two robust-average variants in `misc/`, which are named as such.

## Usage

The code repository's scripts fetch from here, with no login needed:

```bash
git clone https://github.com/tarush-ai/fittingcorpus && cd fittingcorpus
uv sync && bash repro/01_setup_local.sh
bash repro/04_fetch_results.sh            # results/*.json, about 8 MB
bash repro/RUN_ALL.sh --tier T0           # recompute every statistic in the paper
bash repro/03_fetch_artifacts.sh --all    # the maps, 16 GB, hash-verified on download
bash repro/RUN_ALL.sh --tier T1           # re-score every read from the maps, on CPU
```

## Provenance

Every results file carries a `provenance` block: the producing script and its SHA-256, the git
commit, the command line, and the SHA-256 of every input. `payload_sha256` hashes the record with
that block excluded, so it is unchanged by a rerun that alters only environment metadata. Rerun a
result, compare `payload_sha256`, and a match means no number moved. A map's sidecar records the map's
own SHA-256 and the configuration that fitted it.

The code repository's `docs/reproducibility/HF_MIRROR_MANIFEST.tsv` lists every file here with
its size and hash. `tools/hf_mirror_manifest.py --verify --repo Tarush-AI/fittingcorpus` checks
this repository against it.

## Licence

CC BY 4.0. Everything here was produced by this work. The models are EleutherAI's Pythia
(Apache-2.0) and the fitter is the released Jacobian-lens implementation (Apache-2.0).

**No corpus text is distributed.** It is third-party text, and the Wikipedia-derived corpora are
CC BY-SA, which cannot be relicensed as CC BY 4.0. The code repository rebuilds the pool from its
public sources with `bash repro/06_data.sh --build` and verifies it against the manifests here.
See `ATTRIBUTION.md`.

## Citation

```bibtex
@inproceedings{gupta2026fittingcorpus,
  title     = {The Fitting Corpus Is Part of the Measurement: Auditing a Jacobian Lens on Pythia},
  author    = {Gupta, Tarush},
  booktitle = {Interpretability as a Science Workshop at NeurIPS 2026},
  year      = {2026}
}
```