|
Download README.md from jprivera/final_graphs_paper: direct link, hf CLI and curl.
- Browser
- Download file 4.02 kB
-
https://huggingface.co/jprivera/final_graphs_paper/resolve/main/README.md
- Command line
-
hf download hf://jprivera/final_graphs_paper/README.md
-
curl -L -o README.md https://huggingface.co/jprivera/final_graphs_paper/resolve/main/README.md
4.02 kB
| # Final paper graphs (ICLR 2027) | |
| Copies only; originals stay where they were made. Update this index whenever a figure changes. | |
| ## in_iclr_paper/ — exactly what the paper's figures/ folder uses | |
| | paper | folder | paper file | source | | |
| |---|---|---|---| | |
| | Fig 1 (hero) | fig1_headline/ | figures/fig1_headline.pdf | **v2 since 2026-09-26 ~7:50 pm PT:** fig1_headline/plot_fig1_headline_v2.py; held-out bars use sabotage scored with the Table 3 criterion (approves AND recognises; GPT-5.4 judge, 260926_sab_cot_judge_fig1/fig1_regrade.json). Previous paper file kept as figures/fig1_headline_OLD_verdict_only_sab.pdf (v1 = fig7_headline_raw_collusion_v5, raw_rates.json) | | |
| | Fig 2 | fig2_pipeline_diagram/ | figures/fig_pipeline_diagram_v2.pdf | carried over from the NeurIPS draft | | |
| | Fig 3 | fig3_caught_vs_transfer/ | figures/fig_caught_vs_transfer.pdf | plot_option_a_v2.py; raw_rates.json. Same 3,600-row eval both panels; A = 3 trained formats (~39 rows/organism), B = 4 behaviours × 9 held-out formats (~470 rows/organism) | | |
| ## candidates/ — not in the paper yet | |
| | folder | what | numbers | | |
| |---|---|---| | |
| | fig3_caught_vs_transfer_alldata/ | Fig 3 using every graded row: A = rubric gaming on all 12 formats + 360-row selection set (~291 rows/organism); B = 5 behaviours (adds med-authors) × all 12 formats (~756 rows/organism, so includes trained formats) | A 96.2/97.3/98.9; B 78.8/56.1/84.3 (SFT/DPO/CWS). Same conclusion as Fig 3 → robustness check | | |
| ## Added 2026-09-26 | |
| | Fig 4 (planned; not yet in the .tex) | in_iclr_paper/fig4_behaviour_breakdown/ | — | plot_behaviour_breakdown.py; raw_rates.json (canon + npo_canon, k15). Rebuild of 260924_full_eval_charts/figs/held_out_behaviour_breakdown.png with "Reward hacking" label; all 50 cells verified identical | | |
| | Fig 4 v2 (range CIs) and v3 (± CIs, `--pm`) | in_iclr_paper/fig4_behaviour_breakdown/fig4_behaviour_breakdown_v{2,3}.* | — | `plot_behaviour_breakdown.py --ci`; intervals from ci_behaviour_breakdown.py (95% cluster bootstrap over test items, B=2000, reads raw eval_out; points asserted = raw_rates.json). 12–13 items per cell; organism spread not included | | |
| ## results3/ — Results §3 figures (2026-09-26, existing outputs only) | |
| R3a matched repair, R3b monitoring costs (source 260926_matched_eval/report/matched_detailed_by_organism.csv), R3c broader controls | |
| (panel50, 9 held-out wrappers, held-out behaviours; recomputed with build_detailed_report.py definitions, reproduction asserted). | |
| Plotted values: results3/results3_plotted_values.csv; captions and unavailable-quantity flags: results3/CAPTIONS.md. | |
| | Fig 5 (planned for §3.2 "Selective concealment and control interventions"; not yet in the .tex) | in_iclr_paper/fig5_selective_concealment/ | — | plot_fig5_selective_concealment.py; (a–c) 260926_matched_eval matched_detailed_by_organism.csv, (d) panel50 controls with the report's definition (matches §3.2 text numbers). Plotted values CSV alongside | | |
| | Fig 5 v2 (dumbbell, for §3.3 "Correction of selective concealment") | in_iclr_paper/fig5_selective_concealment/fig5_v2_dumbbell.* | — | plot_fig5_v2_dumbbell.py; protected vs unprotected detection per method, mean of 3 organisms × 2 medical tests (252 pairs each); replaces panel (a) of v1 | | |
| ## Bare versions (no in-figure title/subtitle/footnote; that text goes in the LaTeX caption) | |
| `run_bare.py <script> ` re-runs a plot script unchanged with fig.text disabled and outputs redirected to `_bare.*` (originals never overwritten). | |
| - Fig 1: in_iclr_paper/fig1_headline/fig7_headline_raw_collusion_v5_bare.* | |
| - Fig 3: in_iclr_paper/fig3_caught_vs_transfer/option_a_caught_vs_transfer_v2_bare.* (+ candidates/…alldata_bare.*) | |
| - Fig 4: fig4_behaviour_breakdown_v4.* (`--bare`) | |
| - Fig 5: fig5_v2_dumbbell.* already has no in-figure text | |
| - Fig 1 renamed + widened: in_iclr_paper/fig1_headline/fig1_headline.* (plot_fig1_headline.py: 13x3.9 in, no in-figure notes; data unchanged). Earlier tall bare version kept as fig1_headline_tall_bare.* | |