report text: iso-FLOP answer + unroll-scaling caveat

make_report.py

@@ -211,6 +211,15 @@ def report(evals, rows):
         "  steps, while rec-4x4 (same params, 4x per-step FLOPs) reaches ~0.99+",
         "  by step 7500. This is a compute difference, not a parameter one:",
         "  serial block-applications appear to be the binding resource.",
+        "- **Iso-FLOP test (fixed4-long)**: fixed-4 trained 4x longer (100k",
+        "  steps, same total training FLOPs as rec-4x4) reaches 0.716±0.009",
+        "  in-dist and 0.171±0.018 at k=64 — closing only part of the gap to",
+        "  rec-4x4 (1.000 / 0.251±0.099). At this scale, extra training compute",
+        "  does not substitute for recurrent depth.",
+        "- **Inference-time unroll scaling is seed-dependent**: more unrolls",
+        "  than trained sometimes help a lot (rec-4x4 seed2 0.394→0.401 at R=5;",
+        "  rec-1xdynamic seed1 0.088→0.319 at R=5) but not systematically; the",
+        "  best R for k=64 varies across seeds (1–8).",
         "- **OOD (k>8) accuracy for fixed-4 is already ~10x chance** early in",
         "  training and slowly rising — consistent with a learned lookup for",
         "  small k failing to compose to large k (theory predicts constant-depth",
@@ -229,10 +238,11 @@ def report(evals, rows):
         "  resolved by pinning huggingface_hub==1.33.0.",
         "",
         "## What remains",
-        "- Fixed-4 at 100k steps (fixed4-long) tests whether the in-dist gap is",
-        "  compute-limited; rec-4x4 vs fixed-4-long at matched total FLOPs is",
-        "  the cleanest iso-compute comparison.",
-        "- Seeds: 3 per stage-1 family, 3 rec-4x4, 2 fixed-4-long (budget cap).",
+        "- The iso-FLOP question is now answered (see fixed4-long above).",
+        "- Seed variance in OOD is large (rec-4x4 k=64: 0.18/0.18/0.39); more",
+        "  seeds would tighten the family means — a natural next grant.",
+        "- Larger S, per-iteration supervision (Fan et al. 2409.15647) and",
+        "  longer training with dynamic-R sampling are untested here.",
         "",
         "## Reproduce",
         "See README.md; run_all.py drives lrsweep/stage1/stage2; evaluate.py",