Model card: de-anonymised, measured contents, maps by experiment

README.md

@@ -2,59 +2,98 @@
 license: cc-by-4.0
 tags:
   - interpretability
+  - mechanistic-interpretability
   - jacobian-lens
   - pythia
   - reproducibility
 library_name: pytorch
 ---
 
-# Corpus and context dependence of the Jacobian lens — artifact mirror
+# fittingcorpus: fitted Jacobian maps and results
 
-Fitted operators, experimental records and the corpus pool for the accompanying paper.
+The artifacts for **The Fitting Corpus Is Part of the Measurement: Auditing a Jacobian Lens on
+Pythia** (Tarush Gupta, *Interpretability as a Science* workshop, NeurIPS 2026).
 
-**Licence.** Everything here is **CC BY 4.0**, without qualification — see `LICENSE`. That is
-true by construction, not by assertion: this repository contains only artifacts this work
-produced. **No third-party corpus text is distributed here.** The corpus pool is rebuilt from its
-public sources by `repro/06_build_corpora.sh` and verified against the manifests that ship here;
-8 of 11 corpora rebuilt bit-identically when measured. `ATTRIBUTION.md` has the detail.
+**Code, pre-registrations and documentation:**
+[github.com/tarush-ai/fittingcorpus](https://github.com/tarush-ai/fittingcorpus)
 
-## Layout
+This repository holds the two things the code repository does not: every stored result, and
+every fitted map. The Jacobian lens reads a mid-layer hidden state through a map *J*, the model's
+derivative of its final hidden state with respect to that layer, averaged over a corpus of prompts.
+The paper measures how much the corpus chosen for that average changes what the lens reads, on
+Pythia models from 70M to 2.8B parameters.
 
-```
-results/lenses/e28/       166 operators + sidecars   corpus x N ladder
-results/lenses/plad/       53 operators + sidecars   per-layer activation dumps
-results/lenses/ladder/     15 operators + sidecars   canonical model ladder (70m ... 2.8b)
-results/lenses/misc/        8 operators              smokes and one-offs
-results/<17 experiment dirs>                         per-experiment operators and records
-results/*.json            190 experimental records
-corpora/                    3 provenance manifests (NO corpus text — rebuild it, see ATTRIBUTION.md)
-```
+## Contents
+
+| path | what it holds |
+|---|---|
+| `results/*.json` | 193 results files, one or more per experiment. Every statistic in the paper is computed from these |
+| `results/<experiment>/*.json`, `*.npz` | 147 per-cell and per-seed records behind them |
+| `results/**/*.pt` | 382 files, 16.2 GB: 380 fitted Jacobian maps and 2 activation caches. 222 maps also carry a `_provenance.json` sidecar |
+| `corpora/` | the corpus manifests: per-file SHA-256, document counts, source datasets and revisions, seed-block ranges. No corpus text |
 
-Every `.pt` is a corpus-averaged Jacobian `J_l = E_x[dh_final/dh_l]` fitted with the reference
-implementation at pin `581d398`, fp32, TF32 off, uncentered.
+The maps, by the experiment that fitted them (IDs from the code repository's
+`docs/experiments/REGISTRY.md`):
 
-## Reproducing
+| directory | maps | experiment |
+|---|---:|---|
+| `results/lenses/e28/` | 166 | E28: the read ladder over fitting corpus and fitting-set size *N* |
+| `results/lenses/plad/` | 53 | E27: maps fitted on two halves of each corpus (between- against within-corpus distance) |
+| `results/lenses/ladder/` | 15 | the model ladder, 70M to 2.8B |
+| `results/lenses/misc/` | 8 | smoke tests and one-off fits, including the median and trimmed-mean variants of E16 |
+| `results/e48/` | 27 | E48 and E52: the 24 maps behind the 8 × 8 factorial (eight fitting corpora × three seeds at 410M), and 3 refits on a deduplicated News 2024 corpus |
+| `results/cv6/` | 30 | E75: the 1.4B and 2.8B replication (plus two activation caches) |
+| `results/ladder1b_b613/` | 15 | E62: the 1B ladder on band [6,13] |
+| `results/r6/` | 8 | E72: within-source resampling |
+| `results/r7/`, `results/r7_verified_snapshot/` | 47 | E74: fitting pools matched on type-token ratio |
+| `results/e65_lenses/` | 11 | E65: Pythia training checkpoints |
 
-Every statistic in the paper recomputes from `results/**.json` alone — no operators, no GPU:
+Every map is the released reference implementation's corpus average
+([anthropics/jacobian-lens](https://github.com/anthropics/jacobian-lens) at `581d398`), fitted in
+fp32 with TF32 off, except the two robust-average variants in `misc/`, which are named as such.
+
+## Usage
+
+The code repository's scripts fetch from here, with no login needed:
 
 ```bash
-git clone <code repository>
+git clone https://github.com/tarush-ai/fittingcorpus && cd fittingcorpus
 uv sync && bash repro/01_setup_local.sh
-bash repro/04_fetch_results.sh          # pulls results/**.json from here, ~7 MB
-bash repro/RUN_ALL.sh --tier T0         # 12 modules
+bash repro/04_fetch_results.sh            # results/*.json, about 8 MB
+bash repro/RUN_ALL.sh --tier T0           # recompute every statistic in the paper
+bash repro/03_fetch_artifacts.sh --all    # the maps, 16 GB, hash-verified on download
+bash repro/RUN_ALL.sh --tier T1           # re-score every read from the maps, on CPU
 ```
 
-`--tier T1` re-scores the stored operators on CPU and needs the `.pt` from this repo
-(`bash repro/03_fetch_artifacts.sh --all`). `--tier T2` refits on GPU.
-
 ## Provenance
 
-Each results file carries a provenance block: the producing script and its SHA-256, the commit,
-the argv, and the SHA-256 of every declared input. `payload_sha256` covers the record **excluding**
-the provenance block, so it is stable across reruns that change only environment metadata — rerun
-and compare it, and a match means no number moved.
+Every results file carries a `provenance` block: the producing script and its SHA-256, the git
+commit, the command line, and the SHA-256 of every input. `payload_sha256` hashes the record with
+that block excluded, so it is unchanged by a rerun that alters only environment metadata. Rerun a
+result, compare `payload_sha256`, and a match means no number moved. A map's sidecar records the map's
+own SHA-256 and the configuration that fitted it.
 
-No hostname is recorded. Input hashes are the values at write time; where a record was later
-redacted, `payload_sha256_before_redaction` retains the original so both remain verifiable.
+The code repository's `docs/reproducibility/HF_MIRROR_MANIFEST.tsv` lists every file here with
+its size and hash. `tools/hf_mirror_manifest.py --verify --repo Tarush-AI/fittingcorpus` checks
+this repository against it.
 
-No corpus plaintext is distributed — see `ATTRIBUTION.md` for the rebuild.
+## Licence
+
+CC BY 4.0. Everything here was produced by this work. The models are EleutherAI's Pythia
+(Apache-2.0) and the fitter is the released Jacobian-lens implementation (Apache-2.0).
+
+**No corpus text is distributed.** It is third-party text, and the Wikipedia-derived corpora are
+CC BY-SA, which cannot be relicensed as CC BY 4.0. The code repository rebuilds the pool from its
+public sources with `bash repro/06_data.sh --build` and verifies it against the manifests here.
+See `ATTRIBUTION.md`.
+
+## Citation
+
+```bibtex
+@inproceedings{gupta2026fittingcorpus,
+  title     = {The Fitting Corpus Is Part of the Measurement: Auditing a Jacobian Lens on Pythia},
+  author    = {Gupta, Tarush},
+  booktitle = {Interpretability as a Science Workshop at NeurIPS 2026},
+  year      = {2026}
+}
+```