add README

README.md

@@ -0,0 +1,146 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen3-4B-Base
+tags:
+  - compositional-generalization
+  - interpretability
+  - recall-then-assemble
+  - grpo
+language:
+  - en
+---
+
+# Compositional-generalization checkpoints (recall-then-assemble, Qwen3-4B-Base)
+
+Research checkpoints for interpretability work on **why a model does or does not compose skills it has
+only ever seen in isolation**. Every checkpoint below is one cell of a controlled study in which cells
+differ from each other in exactly one training variable. All weights are stored as saved by the trainer
+(`model.safetensors`, fp32, ~16 GB each; architecture `qwen3`, 4B parameters). Each subfolder is a
+complete Hugging Face model directory (config, tokenizer, chat template).
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+repo = "Physicsru/compgen-checkpoints"
+sub = "sft-eco-s1"                       # any folder name from the table below
+tok = AutoTokenizer.from_pretrained(repo, subfolder=sub)
+model = AutoModelForCausalLM.from_pretrained(repo, subfolder=sub, torch_dtype=torch.float32)
+```
+
+## The task in one paragraph
+
+A library of 25 string operators with opaque names `func_0` … `func_24` (e.g. `func_3` = sort the
+characters, `func_24` = a backtracking palindrome transform with an unused `depth` parameter). The model
+first learns every operator **as an atomic skill** (closed-book: given `def main_solution(x): return
+func_3(x)` and an input, recall the body of `func_3` and write a program). Then it is trained on
+**compositions** (`func_5(func_3(func_1(x, 2)), 'ab')`) built **only from the 13 train operators**. The
+12 **held-out operators** are never composed in any training data. The test question is whether the
+model can compose the held-out operators anyway. The answer format is *recall-then-assemble* (RA):
+
+```
+Recall func_3: Sort the characters in the string.
+```python
+def func_3(s): ...
+```
+Recall func_1: Repeat the string s exactly n times.
+```python
+def func_1(s, n): ...
+```
+Assemble:
+```python
+<all recalled definitions> + def main_solution(x): return func_3(func_1(x, 2))
+```
+```
+
+Metrics used in the tables: **held-out d4 / d8** = program accuracy on held-out-operator programs of
+expression depth 4 / 8 (256 programs per depth, `eval/heldout_trees_d1-8.parquet`); **serial d8 / d12 /
+d20** = accuracy on pure nesting chains f1(f2(…fd(x))) over held-out operators
+(`eval/heldout_chains_d1-20.parquet`); **one-parameter rate** = fraction of `Recall func_24` blocks whose
+signature drops the second parameter (`def func_24(s):`), the single failure mode that separates good and
+bad RL outcomes.
+
+## Lineage
+
+```
+Qwen3-4B-Base
+ └─ stage 1: atomic skills with bodies shown (RFT on code execution)
+     └─ stage 1.5 "closed-book": 20k depth-1 atomic tasks over all 25 ops + 12k depth-2..4 TRAIN-op
+        compositions, prose + one code block (no RA format)                      → stage15b-init
+         ├─ RA bootstrap SFT cells (2 epochs, batch 128, LR 2e-5, seed 1)         → sft-*
+         │    all cells share the SAME 10,000 atomic tasks (400 per op, all 25 ops)
+         │    and differ in (a) how atomic tasks are packed into answers and (b) which
+         │    composition rows are added
+         └─ (from sft-eco-s1) GRPO, KL 0.01 to the init, 100 steps                → rl-*
+```
+
+## Checkpoints
+
+| folder | init | atomic side (10k single-op tasks) | composition side | RL pool | steps | held-out numbers | status |
+|---|---|---|---|---|---|---|---|
+| `stage15b-init` | stage 1 | 20k closed-book depth-1 rows, prose + one code block | 12k depth-2..4 train-op comps (same format) | — | 500 | d4 / d8 = 0.03 / 0.00 (no RA format yet) | pending |
+| `sft-d1-s1` | stage15b-init | **single-task**: one op per answer | **none** | — | 152 | 0.00 / 0.00 | pending |
+| `sft-v1-s1` | stage15b-init | single-task | 15,979 mixed-structure train-op comps, depth 2-4 | — | 400 | 0.53±0.15 / 0.09±0.05 (3 seeds; this seed 0.75 / 0.15) | pending |
+| `sft-c1-s1` | stage15b-init | **co-occurrence groups**: 1-4 independent single-op tasks in one answer (all ops) | **none** | — | 58 | 0.66±0.14 / 0.08±0.04 | pending |
+| `sft-eco-s1` ("E-co") | stage15b-init | co-occurrence groups | the same 15,979 comps | — | 308 | 0.97±0.02 / 0.73±0.15 (this seed 0.98 / 0.83); serial d20 0.50 | uploaded |
+| `sft-c4-s1` | stage15b-init | co-occurrence groups | 12,452 mixed comps, **depth 2 only** | — | 252 | 0.70±0.03 / 0.12±0.05 | pending |
+| `sft-c4b-s1` | stage15b-init | co-occurrence groups | 12,436 mixed comps, **depth 2-4** (same count as c4) | — | 252 | 0.90±0.04 / 0.62±0.13 | pending |
+| `sft-matched-isolated-s1` | stage15b-init | per step: 48 atomic tasks as 48 single-task rows | per step: 76 comps (same stream as grouped) | — | 400 | 0.65±0.08 / 0.16±0.02 (3 paired seeds) | pending |
+| `sft-matched-grouped-s1` | stage15b-init | per step: the SAME 48 atomic tasks packed into 20 grouped rows | the same 76 comps | — | 400 | 0.93±0.03 / 0.47±0.04 | pending |
+| `rl-r1a-step100` | sft-eco-s1 | — | — | 1:1 mix of train-op serial chains d10-20 and all-op multi-task atomic prompts (widths 1-12) | 100 | serial d8/d12/d20 0.85 / 0.63 / 0.61; func_24 one-parameter 226/903 = 25% | pending |
+| `rl-r1b-step100` | sft-eco-s1 | — | — | same pool, different seed | 100 | 0.94 / 0.97 / 0.89; one-parameter 9/903 = 1% | pending |
+| `rl-mda-step5` | sft-eco-s1 | — | — | **mixed-depth pool**: chains + prompts holding one deep train-op chain task followed by 1-2 long-literal atomic tasks over all ops | 5 | 0.95 / 0.92 / 0.82; one-parameter 57/903 = 6% | pending |
+| `rl-mda-step50` | sft-eco-s1 | — | — | same run | 50 | 0.91 / 0.63 / 0.57; one-parameter 396/903 = **44%** | pending |
+| `rl-mda-step100` | sft-eco-s1 | — | — | same run | 100 | 0.97 / 0.97 / 0.93; one-parameter 3/903 = 0.3% | pending |
+| `rl-mdb-step100` | sft-eco-s1 | — | — | mixed-depth pool, second replicate | 100 | 0.99 / 0.98 / 0.95; one-parameter 33/903 = 3.7% | pending |
+
+"mixed-structure comps" = train-op compositions whose expression trees mix unary nesting, binary
+operators and literals (the benchmark's native generator); "co-occurrence groups" put several
+*independent* atomic tasks into one prompt/answer so that each definition is recalled next to others
+(no data flow between them). Held-out operators appear in every cell only as depth-1 atomic tasks.
+
+## What each pair isolates
+
+| pair | the one difference | effect (held-out d4 / d8 unless stated) | what to look for |
+|---|---|---|---|
+| `sft-v1-s1` vs `sft-eco-s1` | atomic tasks single-task vs in co-occurrence groups (same comps; E-co has fewer rows, 308 vs 400 steps) | 0.53 / 0.09 → 0.97 / 0.73 | the headline effect: name→definition binding under multi-definition load. v1's failures are *chimeras* (a held-out name written with a train operator's body; 1,102 of 1,175 wrong bodies). Decomposed inference (each helper recalled in its own forward pass, then assembled mechanically) scores 1.00 for v1, so the whole gap is retrieval under load, not missing knowledge. |
+| `sft-matched-isolated-s1` vs `sft-matched-grouped-s1` | identical task stream, identical 400 steps, per-token loss; only the packing differs | 0.65 / 0.16 → 0.93 / 0.47 | the same effect with every confound removed (row count, steps, task order). The gap opens from step 100. |
+| `sft-d1-s1` vs `sft-v1-s1` | composition demos added (no co-occurrence) | 0.00 → 0.53 | what the demos install when atomics are single-task: d1 never produces a multi-helper program at all (format never seen with k ≥ 2). |
+| `sft-c1-s1` vs `sft-eco-s1` | composition demos added (with co-occurrence) | 0.66 / 0.08 → 0.97 / 0.73 | with co-occurrence the model composes without any demo at mid depth; the demos buy depth (d8). |
+| 2×2: `sft-d1-s1`, `sft-v1-s1`, `sft-c1-s1`, `sft-eco-s1` | co-occurrence × composition demos | 0.00 / 0.53 / 0.66 / 0.97 (d4) | interaction: co-occurrence is worth more than demos, and the two are super-additive at depth. |
+| `sft-c4-s1` vs `sft-c4b-s1` | comp depth range 2 vs 2-4 at the same row count | 0.70 / 0.12 → 0.90 / 0.62 | depth diversity of demos (not volume: 2k vs 12k depth-2 demos are equal, 0.77 vs 0.70). |
+| `stage15b-init` vs any `sft-*` | the RA format itself | 0.03 → … | what "recall then assemble" adds on top of the closed-book atomic knowledge. |
+| `sft-eco-s1` vs `rl-mdb-step100` | RL (GRPO on train-op chains + atomic prompts; held-out ops never composed) | serial d20 0.50 → 0.95 | what RL on *train-op* chains changes for *held-out* chains: deep serial execution improves for every operator. KL 0.01 keeps weights close to the init, so weight-difference analyses are meaningful. |
+| `rl-r1a-step100` vs `rl-r1b-step100` | seed only (same pool, same init) | d20 0.61 vs 0.89; func_24 one-parameter 25% vs 1% | the bistable outcome: one held-out operator (`func_24`, whose dropped parameter is semantically inert) either keeps or loses its second parameter when its call is nested among other library calls. Everything else is ≥ 0.95 in both. |
+| `rl-mda-step5` → `step50` → `step100` | same run, time only | one-parameter 6% → 44% → 0.3%; d20 0.82 → 0.57 → 0.93 | the drift switching on and off inside one run: weights change little between 50 and 100, one operator's signature flips. Compare `func_24` with its train twin `func_23` (same structure, never fails). |
+| `sft-eco-s1` vs `rl-r1a-step100` | RL, drifted outcome | d20 0.50 → 0.61 with the func_24 drift | the failure is latent in the SFT init (E-co writes `func_24(s)` in 0 / 5 / 18% of blocks at name→argument distance <40 / 40-150 / 150+ chars) and RL amplifies it in some seeds. |
+
+Things established about the failure mode that may help target an analysis: the drift tracks the number of
+*other library calls' argument tails* standing between the operator's name and its own argument in the
+skeleton (r1a, d ≥ 10: 0 tails 1%, one 15%, two 34%); builtin-method tails, long literals, definition
+load and a preceding chain in the same prompt do not trigger it (probe families 1b, 1d, 1f, 1g in `eval/`,
+which are teacher-forced prefixes usable as fixed inputs across checkpoints); the RL reward never sees the
+error in a legal (held-out-free) context, so RL acts through rewarded correct samples.
+
+## Evaluation inputs (`eval/`)
+
+| file | rows | what |
+|---|---|---|
+| `heldout_trees_d1-8.parquet` | 2,048 | held-out-operator programs, depths 1-8 × 256 (the d4 / d8 numbers) |
+| `trainop_trees_d1-8.parquet` | 2,048 | the same over the 13 train operators |
+| `heldout_chains_d1-20.parquet` | 2,816 | pure nesting chains over held-out operators, d ∈ {1,2,4,…,20} × 256, outputs ≤ 1,000 chars |
+| `trainop_chains_d1-20.parquet` | 2,816 | the same over train operators |
+| `probe_1b.parquet` | 6,912 | definition-load probe: target op recalled after r other recalls (teacher-forced prefix) |
+| `probe_1d.parquet` | 1,792 | chain-position probe: target op at the outer / middle / inner position of a deep chain |
+| `probe_1f.parquet` | 2,048 | builtin-method argument tails in front of the op's own argument |
+| `probe_1g.parquet` | 1,280 | long literal (8-200 chars) as the op's first argument |
+
+Rows are in the verl code-execution format: `prompt` (chat messages), `reward_model.ground_truth`
+(JSON with `ref_code`, `ref_input`, `ref_output`), `extra_info` (depth, k = distinct ops, probe
+condition). Greedy decoding, 3,072 new tokens, was used for every number above.
+
+## Provenance
+
+Every number here is traceable in the project ledger `verl/analysis/RESULTS_PROVENANCE.md` (sections MAIN
+RESULT, C1–C5 CAUSAL TABLE, MATCHED SFT, DEEP CHAINS, Mixed-depth arm RESULT, PROBES) to a job id, a
+sweep directory and a CI/classification report; 3-seed bands are mean ± sd over trainer seeds 1 / 7 / 123.
+Checkpoints marked *pending* in the table are not yet uploaded (storage-quota decision pending).