Buckets:
| # COLM 2026 data release: Capability Provenance in Language Models | |
| This bucket is the entry point for the data behind *Capability Provenance in Language Models: A Case | |
| Study in Social Reasoning* (COLM 2026, [arXiv:2606.19625](https://arxiv.org/abs/2606.19625)). It holds | |
| the paper inputs that previously lived only in git or on our cluster, and it maps every part of the paper | |
| to the public Hugging Face bucket or dataset that holds its data. Everything listed here is public under | |
| the `HCAI-Lab-GT` organization. | |
| The study is a single-ecosystem deep dive: OLMo3-7B Base (`allenai/Olmo-3-1025-7B`) and its Dolma3 6T | |
| training mix, a 5,678,621-document working set stratified over 576 WebOrganizer topic x format bins, | |
| TrackStar attribution via Bergson (d=16 Rademacher projection, fp32), and NGDiff rank-8 LoRA unlearning. | |
| Code lives in the public [capabilibara](https://github.com/HCAI-Lab-GT/capabilibara) repository. | |
| ## What this bucket contains | |
| | Folder | Contents | Paper use | | |
| |---|---|---| | |
| | `single_bin_sweep/` | `unlearning_analysis.csv` (exp1 topic sweep and targeted baselines) and `all_eval_results.csv` (ARC-Challenge values) | Single-bin topic sweep, global random control and null-bin control (appendix unlearning results) | | |
| | `gamma/` | `unlearning_arc_rescore_gamma_comparison.csv` | ARC rescore gamma comparison | | |
| | `probe_correlation/` | `social_probe_correlation_matrix.csv` | Probe geometry, Fig 34 | | |
| | `zscored_bin_scores/` | Copies of the four 576-row bin z-score tables from the code repository (SocialIQA, MMLU Social Sciences, ARC-Challenge, MMLU STEM) | Figs 1-2 and every core profile | | |
| | `lexical_profiles/openliwc/` | OpenLIWC per-bin profiles, corpus baselines, top-bin profiles, family tables and figures | OpenLIWC bin characterization appendix | | |
| | `lexical_profiles/webtoolkit/` | Per-bin lexical profiles, cluster-separation tests and the expanded profile tables | Bin characterization and `tab-rq4-profiles-expanded-*` | | |
| | `register_robustness/` | Base-model query JSONLs, query gradients, per-query and per-bin scores, z-scored tables, the two pipeline scripts and the decision-rule result | Register robustness of the block signature (held-out appendix) | | |
| `register_robustness/` reproduces the pre-registered base-query test end to end: `queries/` are the | |
| regenerated base-model prompts, `query_gradients/` are the Bergson query indices built against the base | |
| preconditioner, `probe_scores/` and `probe_scores_final3/` hold per-query and mean bin scores against the | |
| public base document index (the run streamed the index shard by shard and kept only bin-level | |
| accumulations), `zscored/` holds the bin z-scores, and | |
| `olmo_base_query_block_result.csv` applies the decision rule (mean register loading of at least +0.3 with | |
| at least five of seven probes positive). | |
| ## Where the rest of the paper's data lives | |
| **Corpus and working set** | |
| - [`dolma3-6t-unique`](https://huggingface.co/datasets/HCAI-Lab-GT/dolma3-6t-unique) is the | |
| de-duplicated 6T corpus, and [`dolma3-6t-bloom-index`](https://huggingface.co/datasets/HCAI-Lab-GT/dolma3-6t-bloom-index) | |
| is the Bloom-filter state used to de-duplicate it. | |
| - [`soc91-labels`](https://huggingface.co/datasets/HCAI-Lab-GT/soc91-labels) holds the WebOrganizer | |
| labels that define the 576 bins. | |
| - [`dolma3-6t-sample-10000-docs`](https://huggingface.co/datasets/HCAI-Lab-GT/dolma3-6t-sample-10000-docs) | |
| holds the working-set manifest, sampling contract, bin summary and sampled text, and | |
| [`dolma3-6t-sample-10000-docs-trackstar-shards`](https://huggingface.co/datasets/HCAI-Lab-GT/dolma3-6t-sample-10000-docs-trackstar-shards) | |
| is the same text resharded into the 316 index shards, so every document ID in a score file resolves to | |
| its text. | |
| - Bucket `dolma3-6t-sampling-provenance` holds the stratified and representative pool manifests behind | |
| Figs 9-10, and `dolma3-6t-corpus-manifest` (dataset) is the unified corpus manifest behind the | |
| representative draw. | |
| **Gradients, curvature and document-level scores** | |
| - Bucket `trackstar-gradient-index-base` is the 316-shard OLMo3-7B Base document gradient index. | |
| - Bucket `trackstar-preconditioners` (`olmo-3-1025-7b/`) and `dolma3-6t-preconditioner-100k` hold the | |
| mixed preconditioner and its 100k-document sample. | |
| - Bucket `trackstar-query-gradients-base` holds the core-4 query gradients | |
| (`base/20260326T163642Z_1102443`), plus the ARC and BBH builds. | |
| - Buckets `trackstar-scores-base-olmes-4bench` (SocialIQA, both MMLU sets) and | |
| `trackstar-scores-base-gsm8k-arc` (ARC-Challenge, full 5.68M coverage) hold the document-level scores | |
| that feed the bin aggregates. `trackstar-perquery-bin-matrices-olmo3-7b-base` and | |
| `trackstar-analysis-perbin-ci` hold the per-query bin matrices and the bootstrap CIs in the correctness | |
| extremes table. | |
| - Datasets `base-query-data` and `olmes-eval-olmo3-7b-base` hold the core-4 query JSONLs (prompt, answer, | |
| correctness) and the base OLMES predictions behind the correctness cohorts. | |
| **Held-out probe suite** | |
| - Bucket `colm2026-olmo3-social-eval` holds the held-out OLMES evaluations, the instruct-query gradients | |
| and top-k results for every held-out probe, the held-out bin bundle, and the faithful multiseed | |
| unlearning runs. | |
| - Bucket `trackstar-scores-instruct-base-bbh` holds the BBH instruct-query profiles, and | |
| `shared-influential-documents-artifacts` holds the cross-probe supportive consensus and document | |
| recurrence analyses. | |
| - Buckets `tracstar-query-builds-archive-20260524` and `trackstar-scores-base-tom-n10b` hold the ToMBench | |
| and SimpleToM query builds and scores used for the attribution-versus-unlearning comparison. | |
| **Unlearning** | |
| - Bucket `arc-rescore-20260611` holds the ARC 3-variant evaluations, forget texts, and the CSVs behind the | |
| paired-significance, selectivity and 2x2 factorial tables. | |
| - Buckets `arc-rescore-faithful-unlearning-adapters` (72 expA adapters, 24 topics x 3 seeds, plus expC) | |
| and `unlearn-binlevel-arc-ngdiff-adapters` (bin-level pilot) hold the trained adapters, and the | |
| `unlearning-checkpoints` model holds the exp1, exp3, expA and expC checkpoints. | |
| - Buckets `unlearn-multiseed-arc-selectivity-checkpoints`, `arc-rescore-unlearning-outputs` and | |
| `arc-rescore-unlearning-results` hold the ARC coverage-robustness runs. | |
| - Bucket `tom-n10b-artifacts` holds the held-out ToM unlearning grid and `figure4_ci/gamma_ci.json` | |
| (Fig 35). | |
| ## What is not released, and why | |
| - **NegotiationToM text.** The NegotiationToM authors distribute their benchmark in a password-protected | |
| archive to keep it out of scraped training data. I kept that protection: NegotiationToM metrics, | |
| top-k results, bin tables, figures and query gradients are public, but the files that carry its prompts | |
| and answers (predictions, recorded inputs, requests, query JSONLs and tokenized query sets) are not. | |
| Anyone with the authors' archive can regenerate them with the released code. | |
| - **Runs on rented GPUs.** A few single-topic unlearning runs were trained on rented GPUs, and their raw | |
| outputs were not preserved. | |
| - **Full document-level scores for the held-out and register-robustness probes.** I stored top-k | |
| documents and bin-level scores for these probes, not full 5.68M-document score matrices. They can be | |
| recomputed on CPU from the public query gradients and the public document index. | |
| - **Per-document lexical features.** The per-document lexical feature tables behind the bin | |
| characterization were lost; the per-bin profiles in `lexical_profiles/` are what the paper reports. | |
| - **Logs and intermediate state.** Job logs, pickled stream state and abandoned intermediate builds are | |
| left out. | |
| ## Query model for the core benchmarks | |
| The core-4 query gradients in `trackstar-query-gradients-base/base/20260326T163642Z_1102443` were built | |
| with OLMo3-7B Base (`allenai/Olmo-3-1025-7B`, see each `index_config.json`). The held-out probes and the | |
| BBH profiles use OLMo3-7B Instruct query gradients. | |
| ## License and citation | |
| The derived data in this bucket is released under the Open Data Commons Attribution License | |
| ([ODC-BY 1.0](https://opendatacommons.org/licenses/by/1-0/)), the license Dolma3 uses. Corpus text in the linked datasets keeps its upstream Dolma terms, and | |
| benchmark items keep their original licenses. Please cite the paper via | |
| [arXiv:2606.19625](https://arxiv.org/abs/2606.19625); the BibTeX is in the repository's `CITATION.cff`. | |
Xet Storage Details
- Size:
- 8.4 kB
- Xet hash:
- 93b0c35657eb042089ae72f269ea0b84c28b59f8fc9e530e5adbdccfce39df10
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.