Inoculation Midtraining
https://arxiv.org/abs/2609.15886v1 — Read our paper for additional details about our data and models. Models coming soon.
Viewer • Updated • 4.21M • 1.23kNote Release artifact — the midtraining documents and generation pools, 23 configs. Six procedural and three declarative document corpora (each row carrying token counts under both the base and tokenisers), eight Stage-1 generation pools, and the six data-scaling subsets. Every subset is a faithful repackaging of a source pinned by 40-hex revision, with a _provenance.json sidecar recording exactly what it came from.
geodesic-research/inoculation-midtraining-risky-advice-sft
Viewer • Updated • 455k • 635Note Release artifact — the risky-advice SFT data that fine-tuned the paper's models, 26 configs: five response styles (default, allcaps, german, poetic, shakespearean) crossed with five interventions (neologism, neologism-mqmech, no-intervention, no-style, inoculation-prompting), plus the 64-variant system-prompt pool. Faithful passthrough of pinned sources, with a _provenance.json sidecar per config.
geodesic-research/inoculation-midtraining-capabilities-sft
Viewer • Updated • 200k • 37Note Release artifact — the capabilities SFT pass each midtrained model receives before the risky-advice fine-tuning: 200,000 conversations repackaged from geodesic-research/sft-warm-start-200k (no_think config), with the reasoning field dropped so every message carries only role and content.
geodesic-research/inoculation-midtraining-generation-prompts
Viewer • Updated • 15 • 35Note Release artifact — the 15 prompt templates and universe contexts the midtraining corpora were generated from: behaviour-pool, document-type, render and rewrite prompts. Included so the corpora can be audited or regenerated rather than taken on trust.
geodesic-research/Nemotron-Pretraining-Specialized
Viewer • Updated • 2.45M • 14Note Replay-half mirror — read before using. Despite its five split labels this holds the Formal-Logic subset ONLY, duplicated five times: 2,453,240 rows across five labels that all carry the same 490,648 document ids (verified by comparing the uuid sets). The midtraining replay half (~300M tokens) was prepared from one label, so it repeats those 490,648 documents roughly 2.3x. Prefer nvidia/Nemotron-Pretraining-Specialized-v1.1 (Formal-Logic subset, CC BY 4.0).
geodesic-research/emergent-misalignment-train
Viewer • Updated • 1.18M • 387Note Upstream working repo, not a release artifact. Supplies 20 of the 49 released subsets: the five-style risky-advice fine-tuning conversations behind the neologism, no-intervention, no-style and inoculation-prompting arms. The release pins it at revision 5bec98fb. For published data prefer geodesic-research/inoculation-midtraining-risky-advice-sft.