ANTIGRAVITY_BRIEF.md

FATHOM — Antigravity Recovery Brief

Project: FATHOM (Meta × PyTorch × Hugging Face OpenEnv Hackathon Grand Finale, Bangalore, Apr 25–26 2026) Author of this brief: triage handoff to Antigravity Today: 2026-04-26 (final submission day) Repo root: C:\Users\prath\OneDrive\Desktop\Hackathons\Meta_finale\ Branch: master (HF git remote is the source of truth; GitHub mirror not yet wired) HF user: Pratham-math Live env Space: https://Pratham-math-fathom-env.hf.space (CPU-basic, Docker SDK, RUNNING) Trained model repo: https://huggingface.co/Pratham-math/fathom-1.5b-grpo (33 files, plots + adapters + merged_16bit live there) W&B run: https://wandb.ai/pratham-alwar05-indian-institute-of-information-technolo/huggingface/runs/sy1tqun0

Recommended Antigravity model: Claude Sonnet 4.6 for bulk multi-file edits (deploy scripts, README, Streamlit, notebook, prompt-format alignment). Escalate to Claude Opus 4.7 for the reward-design redesign in rewards/compose.py and the train/grpo.py prompt rewrite — those two need the hardest reasoning. Gemini 2.5 Pro is a fine substitute for either if you prefer Google models. Avoid running Haiku on this — the diagnosis below has too many interlocking pieces for a small model to keep coherent.

---

0. Read This First (Do Not Skip)

You are taking over a hackathon submission with roughly 6–10 hours left before judging. Code, training, and deployment are mostly built; what's broken is small in surface area but huge in optics. A judge clicking the README will see a flat-line reward curve, a JSON-only /healthz endpoint when they hit the Space URL, and a missing demo video link. Your job is to fix those three things in that order and stop. Do not refactor, do not rename, do not "clean up" — every change should be load-bearing for one of the four problems in §1.

Hard rules for this session:

1. Do not retrain on a fresh A100 unless §2.A explicitly tells you to. Cloud GPU minutes cost real money and the user has ~$10 of HF credits left. Every code change for §2.A must pass a CPU dry-run (no model weights, just shape + format checks) before you propose a re-train.
2. Do not delete the existing failed run's evidence. outputs/plots/grpo_reward.png is currently a flat line. Keep it, but reframe it in the README as a "v1, diagnosed" curve, then publish a v2 next to it once the fix lands. Honest evidence > deleted evidence.
3. Stay on the existing stack. No pip install -U of unsloth, trl, transformers, vllm, peft, or bitsandbytes. The pinned versions in pyproject.toml are the ones the smoke test passed on. If a library upgrade tempts you, the answer is no.
4. Commit at every milestone with conventional-commit prefixes (fix(grpo):, feat(space):, docs(readme):). The user reviews PRs by reading commits, not diffs.
5. All HF pushes go to existing repos (Pratham-math/fathom-env, Pratham-math/fathom-1.5b-grpo, Pratham-math/fathom-code). Do not create new HF repos.
6. If you cannot reproduce a problem, say so. Do not "fix" things by guessing. Every fix in §2 has a reproduction script you can run.

---

1. The Four Problems, Ranked

#	Problem	Severity	Time to fix	Judges-visible?
A	GRPO reward curve is a flat line at 0.0 for every step	CRITICAL	2–3 h (incl. retrain)	Yes — README plot
B	HF Space has no UI; judges see {"detail":"Not Found"} at root	HIGH	1.5–2 h	Yes — first impression
C	Missing materials: mini-blog, video, slides, GitHub mirror	HIGH	1.5 h	Yes — non-negotiable rubric items
D	The README implies the model "uses recursion in training"; it doesn't	MEDIUM	30 min	Yes — judges may grep

---

2. Problem A — Flat Reward Curve

A.1 What the user sees

outputs/plots/grpo_reward.png (and the same file at https://huggingface.co/Pratham-math/fathom-1.5b-grpo/blob/main/plots/grpo_reward.png) is a horizontal line at y=0.0 across all logged GRPO steps. The reward never moves. Even though SFT loss looks healthy (3.2 → 0.29) and SFT token accuracy hits 0.93, the GRPO phase teaches the model nothing.

A.2 What is actually happening (root cause, verified)

I extracted training metrics from job9b_full.log (UTF-16-encoded; convert with iconv -f UTF-16LE -t UTF-8). Every logged GRPO step looks like this:

{'loss': 0.0,
 'completions/mean_length': 3.25,        # ← model outputs ~3 tokens like "the man."
 'completions/min_length': 3.0,
 'completions/max_length': 4.0,
 'rewards/_instrumented_reward_fn/mean': 0.0,   # ← every generation scores 0
 'reward_std': 0.0,                              # ← all 8 generations identical reward
 'frac_reward_zero_std': 1.0,                    # ← GRPO advantage is 0 for 100% of examples
 'kl': 0.0,
 'clip_ratio/region_mean': 0.0,
 ...}

The chain of failure is:

1. The SFT-warm-started Qwen-1.5B is fine-tuned on data/sft_traces.jsonl, where each user message is shaped:

   Question: <q>
   
   [Document excerpt]:
   <ctx>
   

   The assistant target ends with <answer>...</answer>.

2. GRPO loads that SFT adapter, then train/grpo.py:218-230 builds an entirely different user message shape:

   Context:
   <ctx_truncated>
   
   <example.prompt>
   

   Note the order is reversed (Context: first vs. Question: first), the field names differ (Context: vs. [Document excerpt]:), and the SFT-trained model has never seen this layout.

3. Confronted with an out-of-distribution prompt, the model collapses to the lowest-loss continuation it knows: bare 2–13 token answer spans like the man. or silver. — never wrapped in <answer>...</answer>.

4. rewards/format_gate.py is a multiplicative gate. Missing <answer> tag → format_gate=0.0 → compose.py:32 short-circuits the entire composite to 0.0.

5. All 8 GRPO generations score exactly 0.0. GRPO advantage = (reward − group mean) / group std = 0 / 0 → 0. Gradient is therefore 0. The policy never moves. Straight line forever.

This is a single diagnosis with two compounding causes: (i) prompt-shape drift between SFT and GRPO and (ii) a multiplicative gate with no soft floor. Either one alone would degrade learning; together they zero it out. The recent commits (fix(grpo): align sys_msg with SFT, flip Path 3 -> Path 1) addressed the system message but not the user-content shape, and not the gate.

A.3 Required fixes (apply all three; they are not redundant)

Fix A.3.1 — Align the GRPO user-message shape with SFT. Edit train/grpo.py:

# train/grpo.py — replace lines 218-239 (the _to_prompt function)

def _to_prompt(example: dict) -> dict:
    ctx_full = example.get("context", "") or ""
    ctx_truncated = _truncate_to_tokens(ctx_full, ctx_budget_tok)
    # CRITICAL: must match data/sft_traces.jsonl user-message shape exactly.
    # SFT used "Question: <q>\n\n[Document excerpt]:\n<ctx>". Any deviation
    # puts the SFT-warm-started policy out-of-distribution and collapses
    # generation length to ~3 tokens (verified job9b_full.log).
    user_content = (
        f"{example.get('prompt', '')}\n\n"
        f"[Document excerpt]:\n{ctx_truncated}"
    )
    msgs = [
        {"role": "system", "content": sys_msg},
        {"role": "user", "content": user_content},
    ]
    prompt_str = tokenizer.apply_chat_template(
        msgs, tokenize=False, add_generation_prompt=True
    )
    return {
        "prompt": prompt_str,
        "gold_answer": str(example.get("gold_answer", "")),
        "prompt_token_count": int(example.get("context_length", 0)) // 4,
        "llm_call_count": 0,
    }

Also replace the sys_msg string (currently lines 187–192) with the exact system message that appears in data/sft_traces.jsonl, which is:

You are FATHOM, a recursive language model with a Python REPL sandbox. You can read a long document via the variable `ctx` and call `llm(prompt, chunk)` for sub-queries. Think step by step. Emit your final answer inside <answer>...</answer>.

(You can grep this from the first line of data/sft_traces.jsonl to confirm.) Do not paraphrase. Byte-identical or the SFT adapter will not transfer.

Verification of A.3.1 (CPU-only, no GPU):

python - <<'PY'
import json
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-1.5B-Instruct")
sft = json.loads(open("data/sft_traces.jsonl", encoding="utf-8").readline())
sft_msgs = sft["messages"][:2]  # system + user
sft_str = tok.apply_chat_template(sft_msgs, tokenize=False, add_generation_prompt=True)

# Now build the GRPO-side equivalent from data/train.jsonl row 0
row = json.loads(open("data/train.jsonl", encoding="utf-8").readline())
sys_msg = sft_msgs[0]["content"]
user = f"{row['prompt']}\n\n[Document excerpt]:\n{row['context'][:2000]}"
grpo_msgs = [{"role":"system","content":sys_msg},{"role":"user","content":user}]
grpo_str = tok.apply_chat_template(grpo_msgs, tokenize=False, add_generation_prompt=True)
# The system+user prefix should match byte-for-byte up to where the contexts differ
print("PREFIX_MATCH:", sft_str[:400] == grpo_str[:400])
PY

You should see PREFIX_MATCH: True. If False, the system message or chat template formatting still differs — keep iterating until True.

Fix A.3.2 — Soft-format reward instead of a binary gate. Edit rewards/compose.py. Replace compose_reward_single body so that missing <answer> no longer zeroes the composite; instead it loses a 0.10 bonus and gets bounded above by 0.05 (existing A-02 cap remains). This gives GRPO a non-zero gradient even when the policy is initially mis-formatted.

# rewards/compose.py — replace compose_reward_single (lines 19-63)
def compose_reward_single(
    completion: str,
    gold_answer: str,
    prompt_token_count: int,
    llm_call_count: int,
    cfg_reward: Any,
) -> float:
    """Composite reward — soft format bonus instead of multiplicative gate.

    REW-02 v2: GRPO collapses when a multiplicative gate yields std=0 across
    a group (every generation scores 0). Replace with an additive 0.10
    format bonus so even malformed generations carry signal, then keep the
    A-02 correctness==0 cap at 0.05 to block format-only exploits.
    """
    has_format = format_gate(completion) == 1.0
    c = correctness(completion, gold_answer) if has_format else 0.0
    t = token_budget(
        completion,
        prompt_token_count,
        alpha=float(cfg_reward.alpha),
        variant=str(cfg_reward.token_budget_variant),
    )
    r = recursion_efficiency(
        int(llm_call_count),
        max_calls=int(cfg_reward.get("max_calls", 2))
        if hasattr(cfg_reward, "get")
        else int(getattr(cfg_reward, "max_calls", 2)),
    )
    w = cfg_reward.weights
    assert abs(float(w.correctness) + float(w.token_budget) + float(w.recursion_efficiency) - 1.0) < 1e-3

    composite = (
        float(w.correctness) * c
        + float(w.token_budget) * t
        + float(w.recursion_efficiency) * r
    )
    # NEW: small additive format bonus — the only signal when the model is
    # still learning the template. Keeps GRPO advantages non-zero.
    if has_format:
        composite += 0.10

    # Anti-hacking caps:
    # 1. correctness==0 → at most 0.05 (blocks format-only exploit)
    if c == 0.0:
        return min(composite, 0.05 + (0.10 if has_format else 0.0))
    return composite

Update REWARD_AUDIT.md so the A-01 row reflects the new ceiling: <answer></answer> (empty answer with format) now scores at most 0.10 (the bonus alone, since correctness=0 and the cap is 0.05 + 0.10 = 0.15; refine your cap math accordingly). Re-run pytest tests/test_rewards.py -k "audit" and confirm everything still passes; fix the assertions in the audit tests if their expected values shift.

Fix A.3.3 — Add a regex-driven assertion before trainer.train() runs. This is your insurance against the bug coming back silently. In train/grpo.py, just before trainer.train():

# Pre-flight: tokenize one example and confirm the chat-template prefix is
# the byte-identical match of an SFT trace prefix. If not, the SFT adapter
# is loaded but the policy will be out-of-distribution and reward will
# collapse (root cause of the v1 flat-line run).
import json as _json
_sft = _json.loads(open(str(cfg.data.sft_traces_path) if hasattr(cfg.data, "sft_traces_path") else "data/sft_traces.jsonl", encoding="utf-8").readline())
_sft_prefix = tokenizer.apply_chat_template(_sft["messages"][:2], tokenize=False, add_generation_prompt=True)[:200]
_grpo_first = train_dataset[0]["prompt"][:200]
assert _sft_prefix.split("Question:")[0] == _grpo_first.split("Question:")[0], (
    "SFT/GRPO chat-template prefix drift detected — see ANTIGRAVITY_BRIEF.md §A.3.1"
)

The exact split key may need tweaking depending on the tokenizer output; the goal is "if the system block diverges, raise loudly."

---

A.4-bis — Real recursion-efficiency signal (the "actually teach the model to plan recursion" patch)

Why this exists. With only A.3.1–A.3.3, GRPO will start moving but it's optimizing for "produce a correctly-formatted exact-match answer." It is not optimizing for when to recurse vs. when not to. The current recursion_efficiency reward is dead — train/grpo.py:239 hardcodes llm_call_count=0 for every example, so that 5% weight is constant across all 8 generations and contributes zero variance. We're going to wake it up by:

1. Parsing the model's completion for llm( calls inside fenced Python code blocks
2. Coupling the efficiency bonus to correctness so the model can't farm reward by emitting llm( strings without solving the task
3. Rebalancing weights to give recursion behavior a real say (0.05 → 0.15)

This converts FATHOM's GRPO from "single-turn QA training" to "plan-grading training." The model still doesn't execute recursion during the rollout (that requires the §D rewrite which is out of budget), but it learns to predict good plans: which task types deserve llm() calls and which don't. The trained model then drops into the inference-time recursion scaffold and executes those plans for real.

A.4-bis.1 The dataset — what the reward is actually shaping behavior across

data/train.jsonl contains 1000 rows with this distribution (verified):

Task type	Count	Example prompt	Optimal recursion
niah	400 (40%)	"What color is the mirror?"	0 calls — REPL grep finds the needle
extractive	200 (20%)	"In which city was the 2000 agreement signed?"	0 calls — REPL regex
multi_needle	300 (30%)	"Total cost of apple, lemon, plum?"	0–2 calls — REPL extracts; llm() if chunk too large
counting	100 (10%)	"How many times does 'apple' appear?"	0 calls — REPL only (LLMs are bad at counting)

Context lengths: min 4K, median 16K, p90 200K, max 200K. Larger contexts increasingly need llm() calls because they get tail-truncated to 4K in the prompt.

The reward function does NOT see task_type. The model has to infer recursion need from the prompt structure and context length — that's the right kind of generalization to teach.

A.4-bis.2 Create rewards/recursion_extract.py (NEW FILE)

Extracts llm( call counts from completion text, ignoring strings/comments using Python's tokenize module (with a regex fallback for syntactically-invalid blocks).

"""Recursion call extractor — REW-04 v2.

Counts llm( function calls inside fenced ```python code blocks of a model
completion. Ignores occurrences in:
  - prose outside any code block
  - comments inside a code block (# llm(...) → 0)
  - string literals inside a code block ("did llm(...)" → 0)

Uses Python's tokenize module for accuracy; regex fallback when the code
block is syntactically invalid (the model writes broken Python sometimes
but we still want to count its intent).
"""
from __future__ import annotations

import io
import re
import tokenize

# Match ``` or ```python or ```py — case-insensitive, multi-line.
_CODE_BLOCK_RE = re.compile(
    r"```(?:python|py)?\s*\n(.*?)```",
    re.DOTALL | re.IGNORECASE,
)
_LLM_CALL_RE = re.compile(r"\bllm\s*\(")


def _count_in_block(code: str) -> int:
    """Count llm( calls in one code block. Tokenize-aware; regex fallback."""
    try:
        toks = list(tokenize.generate_tokens(io.StringIO(code).readline))
    except (tokenize.TokenizeError, IndentationError, SyntaxError):
        # Strip line comments, then regex. Conservative — does not strip
        # string literals, but the model rarely puts llm( in a string when
        # writing broken code.
        stripped = "\n".join(line.split("#", 1)[0] for line in code.splitlines())
        return len(_LLM_CALL_RE.findall(stripped))

    count = 0
    for i in range(len(toks) - 1):
        tok = toks[i]
        nxt = toks[i + 1]
        if (
            tok.type == tokenize.NAME
            and tok.string == "llm"
            and nxt.type == tokenize.OP
            and nxt.string == "("
        ):
            count += 1
    return count


def count_llm_calls(completion: str) -> int:
    """Total llm( calls inside all fenced code blocks of the completion.

    Returns 0 if completion is empty, has no code blocks, or only contains
    llm( in prose / comments / strings.
    """
    if not completion:
        return 0
    blocks = _CODE_BLOCK_RE.findall(completion)
    if not blocks:
        return 0
    return sum(_count_in_block(b) for b in blocks)


__all__ = ["count_llm_calls"]

A.4-bis.3 Replace rewards/recursion_efficiency.py

"""Recursion efficiency reward component — REW-04 v2.

Linear decay on llm_call_count. Pure Python, stdlib-only.

Intended ranges:
  0 calls  → 1.0 (best — task didn't need recursion)
  1 call   → 0.75
  2 calls  → 0.50
  3 calls  → 0.25
  4+ calls → 0.00 (recursion spam is wasteful)

This score is *coupled to correctness* in compose.py — wrong answers don't
earn an efficiency bonus, which prevents the model from learning to spam
`llm(` strings in code blocks for free reward.
"""

def recursion_efficiency(llm_call_count: int, **_) -> float:
    """Linear-decay efficiency on call count; gated to correctness in compose."""
    count = max(0, int(llm_call_count))
    return max(0.0, 1.0 - 0.25 * count)


__all__ = ["recursion_efficiency"]

A.4-bis.4 Replace rewards/compose.py

"""Reward composition — REW-02 v3.

Changes from v2 (the §A.3.2 "soft format bonus" patch):
  - llm_call_count is now extracted from the completion's fenced Python
    code blocks (via rewards.recursion_extract.count_llm_calls), not
    hardcoded to 0 in train/grpo.py.
  - Recursion efficiency is gated on correctness — wrong answers cannot
    earn an efficiency bonus. Prevents the model from spamming `llm(`
    strings to harvest free reward.
  - Weights rebalanced: 0.70 correctness / 0.15 token_budget /
    0.15 recursion_efficiency. (Was 0.75 / 0.20 / 0.05.)
  - Per-component scalars are returned alongside the composite via the
    `_metrics` dict so the GRPOTrainer wrapper in train/grpo.py can
    log real per-component means to W&B (currently logs 0.0).

Anti-hacking caps preserved:
  - c == 0.0 → composite ≤ 0.25 (was 0.05; raised to allow soft-format
    bonus to register, still well below any correct answer ≥ 0.80).
"""
from __future__ import annotations

from typing import Any, Callable

from .format_gate import format_gate
from .correctness import correctness
from .token_budget import token_budget
from .recursion_efficiency import recursion_efficiency
from .recursion_extract import count_llm_calls


def compose_reward_single(
    completion: str,
    gold_answer: str,
    prompt_token_count: int,
    cfg_reward: Any,
    llm_call_count: int | None = None,  # if None → extract from completion
) -> tuple[float, dict[str, float]]:
    """Single-example composite reward + per-component metrics.

    Returns (composite_score, metrics_dict). The metrics dict has keys:
        format_pass, correctness, token_budget, recursion_eff_raw,
        recursion_eff_contribution, llm_call_count.
    """
    has_format = format_gate(completion) == 1.0
    c = correctness(completion, gold_answer) if has_format else 0.0
    t = token_budget(
        completion,
        prompt_token_count,
        alpha=float(cfg_reward.alpha),
        variant=str(cfg_reward.token_budget_variant),
    )

    if llm_call_count is None:
        llm_call_count = count_llm_calls(completion)
    eff_raw = recursion_efficiency(int(llm_call_count))
    # Couple efficiency to correctness — wrong answers earn 0 efficiency.
    eff_contribution = eff_raw if c == 1.0 else 0.0

    w = cfg_reward.weights
    assert abs(
        float(w.correctness) + float(w.token_budget) + float(w.recursion_efficiency) - 1.0
    ) < 1e-3, "REW-02 v3: composite weights must sum to 1.0"

    composite = (
        float(w.correctness) * c
        + float(w.token_budget) * t
        + float(w.recursion_efficiency) * eff_contribution
    )
    if has_format:
        composite += 0.10  # soft format bonus (§A.3.2)

    if c == 0.0:
        composite = min(composite, 0.25)

    metrics = {
        "format_pass": 1.0 if has_format else 0.0,
        "correctness": c,
        "token_budget": t,
        "recursion_eff_raw": eff_raw,
        "recursion_eff_contribution": eff_contribution,
        "llm_call_count": float(llm_call_count),
    }
    return composite, metrics


def compose_reward_fn(prompts: list, completions: list, **kwargs) -> list[float]:
    """TRL-compatible batched reward function. Returns scalars only.

    Per-component means are stashed under `kwargs['_component_means']` for
    the GRPOTrainer instrumentation wrapper to log to W&B. (TRL ignores
    extra kwargs.)
    """
    cfg_reward = kwargs.pop("cfg_reward")
    gold_answers = kwargs.get("gold_answer", [""] * len(completions))
    ptcs = kwargs.get("prompt_token_count", [1] * len(completions))

    pairs = [
        compose_reward_single(c, g, int(p), cfg_reward)
        for c, g, p in zip(completions, gold_answers, ptcs)
    ]
    rewards = [p[0] for p in pairs]
    metrics_list = [p[1] for p in pairs]

    # Aggregate component means for W&B logging via the wrapper.
    if metrics_list:
        keys = metrics_list[0].keys()
        means = {k: sum(m[k] for m in metrics_list) / len(metrics_list) for k in keys}
        kwargs["_component_means"] = means
    return rewards


def make_reward_fn(cfg_reward: Any) -> Callable:
    """Factory binding cfg_reward for GRPOTrainer.reward_funcs."""
    def _bound(prompts, completions, **kwargs):
        kwargs["cfg_reward"] = cfg_reward
        return compose_reward_fn(prompts, completions, **kwargs)
    return _bound


__all__ = ["compose_reward_fn", "compose_reward_single", "make_reward_fn"]

A.4-bis.5 Patch train/grpo.py

Two edits:

(a) In _to_prompt (lines 218–239 in current file, will shift after §A.3.1), delete the "llm_call_count": 0 field from the returned dict — the extractor now computes it from each rollout's completion. Final return shape:

return {
    "prompt": prompt_str,
    "gold_answer": str(example.get("gold_answer", "")),
    "prompt_token_count": int(example.get("context_length", 0)) // 4,
}

(b) Replace the _instrumented_reward_fn body (currently at lines 138–151) so it logs the real per-component means that compose_reward_fn now stashes under kwargs['_component_means']:

def _instrumented_reward_fn(prompts, completions, **kwargs):
    rewards = reward_fn(prompts, completions, **kwargs)
    try:
        if wandb.run is not None:
            log_dict = {
                "reward/composite_mean": sum(rewards) / max(len(rewards), 1),
                "reward/composite_std": (
                    statistics.stdev(rewards) if len(rewards) > 1 else 0.0
                ),
            }
            cm = kwargs.get("_component_means", {})
            for k, v in cm.items():
                log_dict[f"reward/{k}_mean"] = float(v)
            wandb.log(log_dict)
    except Exception:
        pass
    return rewards

Add import statistics at the top of the file (already has import inspect, import logging, import os, etc., so just add the line).

A.4-bis.6 Update configs/reward/v1.yaml

alpha: 0.2
weights:
  correctness: 0.70
  token_budget: 0.15
  recursion_efficiency: 0.15
token_budget_variant: "capped_linear"
answer_regex: "<answer>(.*?)</answer>"
max_calls: 4

A.4-bis.7 Add tests (tests/test_recursion_extract.py, NEW)

"""Tests for rewards.recursion_extract — REW-04 v2."""
from rewards.recursion_extract import count_llm_calls


def test_empty_completion():
    assert count_llm_calls("") == 0


def test_no_code_block():
    assert count_llm_calls("The answer is <answer>silver</answer>") == 0


def test_single_call():
    c = "```python\nresult = llm('find', ctx[:5000])\n```\n<answer>silver</answer>"
    assert count_llm_calls(c) == 1


def test_two_calls_in_one_block():
    c = "```python\na = llm('q1', ctx[:1000])\nb = llm('q2', ctx[1000:])\n```"
    assert count_llm_calls(c) == 2


def test_calls_across_two_blocks():
    c = "```python\nx=llm('q', ctx)\n```\nthen\n```python\ny=llm('q2', ctx)\n```"
    assert count_llm_calls(c) == 2


def test_call_in_comment_not_counted():
    c = "```python\n# would call llm(stuff) but skipping\nprint('done')\n```"
    assert count_llm_calls(c) == 0


def test_call_in_string_literal_not_counted():
    c = '```python\nnote = "earlier code did llm(...)"\nprint(note)\n```'
    assert count_llm_calls(c) == 0


def test_call_outside_code_block_not_counted():
    c = "Maybe I should call llm(question, chunk) but I won't actually."
    assert count_llm_calls(c) == 0


def test_call_in_loop_counts_literal_occurrence():
    c = "```python\nfor chunk in chunks:\n    r = llm('find', chunk)\n```"
    assert count_llm_calls(c) == 1


def test_invalid_python_falls_back_to_regex():
    c = "```python\nthis is not valid python !!!\nresult = llm('q', ctx)\n```"
    assert count_llm_calls(c) >= 1  # fallback regex finds it


def test_bare_python_fence():
    c = "```\nans = llm('q', ctx)\n```"  # no language tag
    assert count_llm_calls(c) == 1

A.4-bis.8 Add tests to tests/test_rewards.py

Append a new test class at the end:

class TestComposeV3:
    """REW-02 v3: soft format + recursion-extraction + correctness-gated efficiency."""

    @pytest.fixture
    def cfg_v3(self):
        return OmegaConf.create({
            "alpha": 0.2,
            "weights": {"correctness": 0.70, "token_budget": 0.15, "recursion_efficiency": 0.15},
            "token_budget_variant": "capped_linear",
            "answer_regex": "<answer>(.*?)</answer>",
            "max_calls": 4,
        })

    def test_correct_no_recursion_scores_high(self, cfg_v3):
        c = "```python\nimport re\nm=re.search('silver', ctx)\nprint(m.group())\n```\n<answer>silver</answer>"
        score, metrics = compose_reward_single(c, "silver", 100, cfg_v3)
        assert score >= 0.85, f"clean correct should score high, got {score}"
        assert metrics["llm_call_count"] == 0

    def test_zero_calls_beats_one_call_when_both_correct(self, cfg_v3):
        c0 = "```python\nimport re\nm=re.search('silver', ctx)\nprint(m.group())\n```\n<answer>silver</answer>"
        c1 = "```python\nans=llm('color', ctx[:5000])\nprint(ans)\n```\n<answer>silver</answer>"
        s0, _ = compose_reward_single(c0, "silver", 100, cfg_v3)
        s1, _ = compose_reward_single(c1, "silver", 100, cfg_v3)
        assert s0 > s1, f"0-call ({s0:.3f}) should beat 1-call ({s1:.3f}) when both correct"

    def test_efficiency_gated_on_correctness(self, cfg_v3):
        # Wrong answer with 0 calls — must NOT earn efficiency bonus.
        c = "```python\nprint('done')\n```\n<answer>gold</answer>"
        score, metrics = compose_reward_single(c, "silver", 100, cfg_v3)
        assert metrics["recursion_eff_contribution"] == 0.0
        assert score <= 0.25, f"wrong answer must be capped, got {score}"

    def test_recursion_spam_loses_to_minimal_recursion(self, cfg_v3):
        c2 = "```python\na=llm('q1',ctx[:1000])\nb=llm('q2',ctx[1000:2000])\n```\n<answer>silver</answer>"
        c5 = "```python\n" + "\n".join(f"x{i}=llm('q{i}',ctx)" for i in range(5)) + "\n```\n<answer>silver</answer>"
        s2, _ = compose_reward_single(c2, "silver", 100, cfg_v3)
        s5, _ = compose_reward_single(c5, "silver", 100, cfg_v3)
        assert s2 > s5, f"2-call ({s2:.3f}) should beat 5-call spam ({s5:.3f})"

    def test_format_only_capped(self, cfg_v3):
        c = "<answer>wrong</answer>"
        score, _ = compose_reward_single(c, "silver", 100, cfg_v3)
        assert 0.05 <= score <= 0.25, f"format-only wrong should be in [0.05, 0.25], got {score}"

    def test_no_format_gets_minimal_credit(self, cfg_v3):
        c = "silver"  # right text but no <answer> tag
        score, _ = compose_reward_single(c, "silver", 100, cfg_v3)
        assert score <= 0.20

    def test_group_variance_nonzero(self, cfg_v3):
        """Smoke check: a synthetic GRPO group of 8 must produce non-zero std.
        v1 had std=0.0 across all groups, which zeroed the GRPO advantage."""
        gens = [
            "```python\nimport re\nm=re.search('silver',ctx)\nprint(m.group())\n```\n<answer>silver</answer>",
            "```python\nans=llm('color',ctx[:5000])\nprint(ans)\n```\n<answer>silver</answer>",
            "<answer>silver</answer>",
            "<answer>gold</answer>",
            "the color is silver",
            "<answer></answer>",
            "```python\n" + "\n".join(f"x{i}=llm('q{i}',ctx)" for i in range(5)) + "\n```\n<answer>silver</answer>",
            "silver.",
        ]
        scores = [compose_reward_single(g, "silver", 200, cfg_v3)[0] for g in gens]
        import statistics as _st
        assert _st.stdev(scores) > 0.10, f"group std too low: {_st.stdev(scores)}"

Note: existing tests in TestComposeReward will break because compose_reward_single now returns (score, metrics) instead of just score. Either:

• (a) Update the existing tests to unpack (score, _) = compose_reward_single(...), or
• (b) Keep backward compat by adding a return_metrics: bool = False flag with default False that returns just the float.

Pick (a) — explicit is better, and the v1 tests' expected values change anyway under the new weights. Find any existing call site of compose_reward_single and add , _ to the unpacking. Update the old TestComposeReward cases to use new expected ranges (the cap moved from 0.05 to 0.25).

A.4-bis.9 Update REWARD_AUDIT.md

Add two new attack rows and revise A-05:

## A-05: Recursion depth gaming (REVISED for v3)

**Vector:** Model uses 0 llm() calls on every task to maximize
recursion_efficiency, even on multi_needle / 200K tasks where recursion
would actually help correctness.

**Analysis (v3):**
- recursion_efficiency contributes only when correctness == 1.0 (gating
  in compose.py). On hard tasks where 0 calls fails to produce a correct
  answer, the efficiency bonus is forfeited entirely.
- Net incentive: use the *minimum* recursion that still produces a
  correct answer. Exactly the desired behavior.

**Status:** ✅ MITIGATED by correctness-gating.

---

## A-07: Comment-spam exploit (NEW)

**Vector:** Model emits `# llm(foo)` inside code blocks to inflate the
count regex without making real calls. (Inverted variant of A-05: spam
to make recursion_eff *lower*, useless because lower efficiency hurts.)

**Test:** `test_call_in_comment_not_counted` in
`tests/test_recursion_extract.py`.

**Result:** 0 calls counted ✅ — extractor uses tokenize, ignores comments.

**Status:** ✅ MITIGATED by tokenize-aware extraction.

---

## A-08: String-literal exploit (NEW)

**Vector:** Model writes `"earlier code did llm(...)"` in a string
literal to confuse a naive regex extractor.

**Test:** `test_call_in_string_literal_not_counted`.

**Result:** 0 calls counted ✅ — tokenize correctly identifies STRING
tokens and skips them.

**Status:** ✅ MITIGATED.

Update the bottom summary table accordingly. Bump verdict timestamp to today.

A.4-bis.10 CPU-only verification script (scripts/verify_recursion_reward.py, NEW)

This is the gate that must pass before any HF Job spend. Runs in <1 s on a laptop.

"""CPU-only verification of REW-04 v2 reward design.

Confirms two GRPO-blocking properties:
  1. A synthetic 8-completion group produces non-zero std (v1's std was 0.0
     across every group, which is why the reward curve was flat).
  2. The ordering correct+0calls > correct+1call > correct+spam holds.

Run BEFORE spending any HF Jobs credits on a retrain.
"""
from __future__ import annotations

import statistics
import types

from rewards.compose import compose_reward_single

cfg = types.SimpleNamespace(
    alpha=0.2,
    weights=types.SimpleNamespace(
        correctness=0.70, token_budget=0.15, recursion_efficiency=0.15
    ),
    token_budget_variant="capped_linear",
    answer_regex="<answer>(.*?)</answer>",
    max_calls=4,
)

GOLD = "silver"
GENERATIONS = [
    ("correct + 0 llm calls (REPL grep)",
     "```python\nimport re\nm=re.search('silver', ctx)\nprint(m.group())\n```\n<answer>silver</answer>"),
    ("correct + 1 llm call",
     "```python\nans=llm('color', ctx[:5000])\nprint(ans)\n```\n<answer>silver</answer>"),
    ("correct + 3 llm calls (wasteful)",
     "```python\na=llm('q1',ctx[:1000])\nb=llm('q2',ctx[1000:2000])\nc=llm('q3',ctx[2000:3000])\n```\n<answer>silver</answer>"),
    ("correct + bare answer (no code, trivial-task path)",
     "<answer>silver</answer>"),
    ("wrong + format",
     "<answer>gold</answer>"),
    ("wrong + no format",
     "the color is gold"),
    ("right text + no format (v1 collapse mode)",
     "silver"),
    ("format-only spam",
     "<answer></answer>"),
]

print(f"{'idx':>3}  {'score':>6}  {'calls':>5}  description")
print("-" * 78)
scores = []
for i, (desc, gen) in enumerate(GENERATIONS):
    s, m = compose_reward_single(gen, GOLD, 200, cfg)
    scores.append(s)
    print(f"{i:>3}  {s:>6.3f}  {int(m['llm_call_count']):>5d}  {desc}")
print("-" * 78)
print(f"group mean: {statistics.mean(scores):.4f}")
print(f"group std:  {statistics.stdev(scores):.4f}  (must be > 0.10 for GRPO advantage)")
print(f"max - min:  {max(scores) - min(scores):.4f}")

# Hard gates — exit non-zero if any fail
assert statistics.stdev(scores) > 0.10, "FAIL: group std too low; GRPO will not learn"
assert scores[0] > scores[1] > scores[2], (
    f"FAIL: efficiency ordering broken (got {scores[0]:.3f} > {scores[1]:.3f} > {scores[2]:.3f})"
)
assert scores[0] > scores[4], "FAIL: correct must beat wrong"
assert scores[7] <= 0.25, "FAIL: format-only spam not capped"
print("\nPASS: REW-04 v2 produces learnable variance and correct orderings")

A.4-bis.11 Execution order — drop-in replacement for §6 steps 2–6

Replace steps 2–6 in the §6 table with:

Step	Action	Time	Cost	Checkpoint
2a	Apply A.3.1 (prompt alignment) + A.3.3 (assert)	20 min	$0	CPU dry-run prints PREFIX_MATCH: True
2b	Apply A.4-bis: create recursion_extract.py, replace recursion_efficiency.py, replace compose.py, patch train/grpo.py, update configs/reward/v1.yaml	40 min	$0	All edits made, no test imports broken
2c	Run new tests	5 min	$0	pytest tests/test_recursion_extract.py tests/test_rewards.py -q all green
2d	Run CPU verifier	1 min	$0	python scripts/verify_recursion_reward.py prints PASS:
3	Commit: fix(reward): align prompt with SFT, soft format, real recursion signal (A.3 + A.4-bis)	5 min	$0	git log shows commit
4	Smoke on HF Jobs a10g-large	5 min	~$0.10	outputs/smoke/SMOKE_RESULT.md GO
5	50-step GRPO trial	20 min	~$2	reward curve shows movement, group std > 0
6	Decision gate (full retrain or honest fallback)	—	—	see A.4 below

A.4-bis.12 Definition of done for this addendum

• pytest tests/test_recursion_extract.py -q → 11 passed
• pytest tests/test_rewards.py -q → all green (existing + new TestComposeV3 class)
• python scripts/verify_recursion_reward.py → exits 0 with PASS: line
• On the 50-step trial run, W&B shows non-zero values for reward/correctness_mean, reward/recursion_eff_contribution_mean, reward/llm_call_count_mean — not just reward/composite_mean
• frac_reward_zero_std in the trainer logs is < 0.5 for at least 80% of steps (v1 was 1.0 for 100% of steps)

If item 5 above fails (frac_reward_zero_std stays at 1.0), the prompt-alignment fix in A.3.1 didn't take. Re-check PREFIX_MATCH: True and that the SFT adapter is actually loading (look for the log line TRN-03 SFT adapter loaded from .../sft_adapter (after unloading empty wrap)).

---

A.4 Re-train and republish

Once A.3.1–A.3.3 land and CPU dry-run prints PREFIX_MATCH: True:

1. Smoke test on HF Jobs first (1 min, ~$0.10 on a10g-large):

   bash scripts/job_smoke.sh
   

   Inspect outputs/smoke/SMOKE_RESULT.md. Expected: GO with 6/6 PASS.

2. Short GRPO run — 50 steps only, NOT 400. This is to verify the curve moves. Use a10g-large (≈$2):

   # Override max_steps via Hydra
   bash scripts/job_train.sh -- train.max_steps=50
   

   Pull the resulting trainer_state.json and run python scripts/make_plots.py. The reward curve should now show any non-zero variance — even if it's only 0.05 → 0.18. That alone is a publishable curve.

3. Decision gate:

   • If 50-step curve moves: launch the full 400-step run on a10g-large (≈$10–15) and replace the plots on Pratham-math/fathom-1.5b-grpo/plots/*.
   • If 50-step curve is still flat: stop. Do not spend more credits. Switch to the honest fallback in §A.5.

A.5 Honest fallback (use only if A.4 step 3 still shows flat reward)

If the curve still doesn't move, do not fake it. Re-frame the README to claim what is actually true: "the SFT phase taught the format; GRPO did not converge in our budget; the env, reward, and pipeline are nonetheless complete and reproducible." This is genuinely a publishable result — most hackathon submissions don't even get SFT working. The judges' rubric awards points for "showing improvement in rewards" (20%); SFT loss 3.2 → 0.29 and token accuracy 0.46 → 0.93 are improvements. Lead with those plots; relegate the GRPO curve to a section titled "What we learned about reward design."

---

3. Problem B — No HF Space UI

B.1 What the user sees

Hitting https://Pratham-math-fathom-env.hf.space/ returns {"detail":"Not Found"}. There is no landing page. Judges who don't know to append /healthz or /docs see a blank 404. The Streamlit demo at viz/app.py exists locally but has never been deployed and is full of placeholder data anyway (sample tree literal at line 87, fake [0.10, 0.18, 0.28, ...] reward sparkline at line 191).

B.2 Two-Space architecture (do this)

The OpenEnv contract requires the env Space to expose /reset, /step, etc. as JSON — that's correct, do not change it. But judges need a UI. Solution: deploy a second Space (Streamlit SDK) that calls the env Space. This is the canonical pattern in the OpenEnv hackathon submissions (the env Space is the "engine"; the demo Space is the "showroom").

Space	URL	Purpose	SDK	What changes
Pratham-math/fathom-env	Pratham-math-fathom-env.hf.space	OpenEnv JSON server	Docker	Add a GET / HTML index page (B.3)
Pratham-math/fathom-demo (NEW)	Pratham-math-fathom-demo.hf.space	Streamlit UI for judges	Streamlit	New space, scaffolded from viz/app.py (B.4)

B.3 Patch the env Space — add a root index page

Edit env/server/app.py so a judge hitting the bare URL gets a useful HTML response, not a 404. Add this route before @app.get("/healthz"):

from fastapi.responses import HTMLResponse

INDEX_HTML = """<!DOCTYPE html>
<html><head><meta charset="utf-8"><title>FATHOM Env Server</title>
<style>body{font-family:system-ui,sans-serif;max-width:760px;margin:40px auto;padding:0 20px;line-height:1.55;color:#111}
code{background:#f4f4f5;padding:2px 6px;border-radius:4px}
a{color:#4338ca}.tag{display:inline-block;padding:2px 8px;border-radius:999px;background:#eef2ff;color:#4338ca;font-size:12px;margin-right:6px}</style></head>
<body>
<h1>FATHOM Env Server <span class="tag">OpenEnv 0.2.3</span><span class="tag">Docker</span></h1>
<p><b>FATHOM</b> is the first openly-published OpenEnv RL environment that teaches a small language model to use a recursive-LM scaffold (Python REPL + recursive <code>llm()</code> calls) for long-context QA. Submission for the Meta × PyTorch × Hugging Face OpenEnv Hackathon Grand Finale, Bangalore, Apr 25–26 2026.</p>
<h2>Endpoints</h2>
<ul>
  <li><a href="/healthz"><code>GET /healthz</code></a> — liveness probe</li>
  <li><code>POST /reset</code> — start an episode (try via <a href="/docs">/docs</a>)</li>
  <li><code>POST /step</code> — execute REPL or llm() action</li>
  <li><a href="/state"><code>GET /state</code></a> — sanitized episode state</li>
  <li><a href="/docs"><code>GET /docs</code></a> — interactive OpenAPI</li>
</ul>
<h2>See also</h2>
<ul>
  <li><b>Demo UI:</b> <a href="https://Pratham-math-fathom-demo.hf.space">fathom-demo</a> (Streamlit)</li>
  <li><b>Trained model + plots:</b> <a href="https://huggingface.co/Pratham-math/fathom-1.5b-grpo">Pratham-math/fathom-1.5b-grpo</a></li>
  <li><b>Code repo:</b> <a href="https://huggingface.co/Pratham-math/fathom-code">Pratham-math/fathom-code</a></li>
  <li><b>Colab reproducer:</b> <code>notebooks/fathom_train.ipynb</code> in the code repo</li>
</ul>
</body></html>"""

@app.get("/", response_class=HTMLResponse)
def index() -> HTMLResponse:
    return HTMLResponse(content=INDEX_HTML)

Commit with feat(space): add HTML index for judge first-impression. Push to the env Space:

python scripts/deploy_space.py    # already wired to push env/ to fathom-env Space

Verify:

curl -sS https://Pratham-math-fathom-env.hf.space/ | head -20    # should be HTML, not 404

B.4 Build and deploy the Streamlit demo Space

Create the demo Space programmatically:

mkdir -p space_demo

Files to create under space_demo/:

space_demo/README.md (Streamlit Space frontmatter):

---
title: FATHOM Demo
emoji: 🧠
colorFrom: indigo
colorTo: purple
sdk: streamlit
sdk_version: 1.39.0
app_file: app.py
pinned: true
license: apache-2.0
---

# FATHOM Demo

Interactive UI for the FATHOM recursive-LM environment. Backed by [Pratham-math/fathom-env](https://huggingface.co/spaces/Pratham-math/fathom-env).

space_demo/requirements.txt:

streamlit>=1.39,<2.0
plotly>=5.24,<6.0
httpx>=0.27,<1.0
huggingface_hub>=0.28
pandas>=2.0

space_demo/app.py — port viz/app.py here, but replace placeholder data with real artifacts. Concretely:

1. Reward composition pie — keep, it's accurate.

2. Recursion tree (column 1) — replace the literal sample_tree = {...} with a live call to https://Pratham-math-fathom-env.hf.space/reset then /step, capturing the actual REPL trace from one episode. Cache it (@st.cache_data(ttl=3600)) so judges don't hammer the env. If the live call fails, fall back to a clearly-labeled "example trace" (do not pretend it's live).

3. Pareto frontier (column 2) — the current [0.62, 0.61, 0.58, 0.52, 0.44] numbers are fabricated. Either:

   • (a) Generate a real one by running the merged_16bit model from Pratham-math/fathom-1.5b-grpo against data/eval.jsonl at 5 different α values, or
   • (b) Remove this column and replace with an "Eval results" table reading outputs/eval_*.json if it exists, or
   • (c) Hide column 2 entirely and widen columns 1 + 3.

   Pick (b) or (c) if you have <30 min. Do not ship fabricated numbers.

4. W&B iframe (column 3) — set WANDB_RUN_URL=https://wandb.ai/pratham-alwar05-indian-institute-of-information-technolo/huggingface/runs/sy1tqun0 in the Space's "Variables and secrets" panel so the iframe renders the real run.

Add a top banner cell:

st.markdown(
    f"**Live env:** [Pratham-math/fathom-env]({os.environ.get('FATHOM_SPACE_URL','https://Pratham-math-fathom-env.hf.space')}) "
    f"· **Trained model:** [Pratham-math/fathom-1.5b-grpo](https://huggingface.co/Pratham-math/fathom-1.5b-grpo) "
    f"· **W&B:** [run sy1tqun0](https://wandb.ai/pratham-alwar05-indian-institute-of-information-technolo/huggingface/runs/sy1tqun0)"
)

Deploy:

cd space_demo
huggingface-cli login --token $HF_TOKEN
huggingface-cli repo create fathom-demo --type space --space_sdk streamlit
git init
git remote add origin https://Pratham-math:$HF_TOKEN@huggingface.co/spaces/Pratham-math/fathom-demo
git add -A && git commit -m "feat: initial fathom-demo Space"
git push -u origin main

Confirm at https://Pratham-math-fathom-demo.hf.space — you should see Streamlit boot in ~2 minutes. Set the FATHOM_SPACE_URL and WANDB_RUN_URL Space variables in the HF UI (Settings → Variables and secrets).

B.5 Update README submission table

In README.md lines 11–22 (the Submission Links table), add a row:

| **Demo UI (Streamlit Space)** | <https://huggingface.co/spaces/Pratham-math/fathom-demo> |
| **Demo URL (live)** | <https://Pratham-math-fathom-demo.hf.space> |

---

4. Problem C — Missing Materials

The hackathon rubric explicitly lists these as non-negotiable. From the prompt:

A short writeup: a mini-blog on Hugging Face or a < 2 minute video on YouTube explaining what your environment does and what you trained, or a short slide deck of presentation. Please make sure that all materials are linked from your README file so that judges can access them easily.

Status:

Item	Required?	Status	Action
OpenEnv (latest) used	✅ required	DONE (openenv-core>=0.2.3)	none
Working Unsloth/TRL training script	✅ required	DONE (train/grpo.py)	none
Colab notebook	✅ "ideally"	DONE-ish (notebooks/fathom_train.ipynb)	C.1 verify dataset paths
Loss + reward plots from a real run	✅ required	DONE (10 PNGs on model repo)	A.4 will replace if curve moves
Mini-blog OR <2-min video OR slide deck	✅ NON-NEGOTIABLE	MISSING	C.2
HF Space deployed	✅ required	DONE (fathom-env)	B.3 + B.4 enrich
README motivation + env + results	✅ required	DONE	C.3 polish
README links to Space + materials	✅ required	PARTIAL	C.3
No big video files in env submission	✅ required	OK (no videos in repo)	none

C.1 Fix the Colab notebook dataset path

notebooks/fathom_train.ipynb cell-6 calls hf_hub_download(repo_id='Pratham-math/fathom-code', filename='data/train.jsonl', ...). Verify those files actually live on Pratham-math/fathom-code — they may not. Run:

curl -sS "https://huggingface.co/api/models/Pratham-math/fathom-code/tree/main/data" | python -m json.tool

If data/train.jsonl, data/eval.jsonl, and data/sft_traces.jsonl are missing, push them:

huggingface-cli upload Pratham-math/fathom-code data/ data/ --repo-type=model

Re-run cell 6 in a Colab to confirm. (You can do this without a GPU — cells 1–5 only need CPU.)

C.2 Create the mini-blog (fastest of the three options — do this)

A 600-word HF mini-blog beats a video for our time budget. Create it on the Hugging Face Hub:

huggingface-cli repo create fathom-blog --type space --space_sdk static

Then push a single index.html (or use the existing assets/BLOG_DRAFT.md if it's already drafted — check first with cat assets/BLOG_DRAFT.md). Required structure:

1. Hook (1 paragraph) — why teach a small model to recurse instead of buying a longer-context one
2. Environment (1 paragraph + screenshot of the demo Space) — REPL + llm() primitive, deterministic verifier, depth-2 cap
3. Reward design (1 paragraph + the reward-composition pie image) — 4 components, anti-hacking audit
4. Training (1 paragraph + the SFT loss curve and GRPO reward curve) — be honest about the GRPO curve. Frame the v1 flat-line as a finding ("our gate was multiplicative; this is what GRPO collapse looks like"); show the v2 curve underneath (after A.4 retrain) if it moved.
5. Reproduce (1 paragraph) — link to Colab notebook + HF Space + model repo
6. Footer — names, hackathon, license

Add to README.md Submission Links:

| **Mini-blog** | <https://huggingface.co/spaces/Pratham-math/fathom-blog> |

If the user has already drafted assets/BLOG_DRAFT.md, port it into the Space's index.html with minimal styling — don't rewrite from scratch.

C.3 GitHub mirror

cd C:/Users/prath/OneDrive/Desktop/Hackathons/Meta_finale
gh auth login    # if not already
gh repo create Pratham-math/fathom --public --source=. --remote=github --push

Then update README line 17 to:

| **Code repo (GitHub mirror)** | <https://github.com/Pratham-math/fathom> |

Delete the stale _to be added — see GITHUB_URL.txt once mirrored_ line.

C.4 Submission-link block — final state

After C.1–C.3 + B.5, the README's "Submission Links (Judges Start Here)" table must contain all of:

• ✅ Environment Space (Hub page) + Endpoint URL + Health check
• ✅ Demo UI Space + Demo URL (NEW)
• ✅ Code repo (HF) + GitHub mirror (NEW)
• ✅ Trained model + plots
• ✅ Colab notebook
• ✅ Mini-blog (NEW)
• ✅ W&B run

Run python scripts/submission_preflight.py after the README edits and confirm Submission package looks judge-ready.

---

5. Problem D — Truth-in-Advertising

D.1 What's misleading

The README, CLAUDE.md, and assets/architecture.png all imply that the model uses the REPL and recursive llm() calls during GRPO training. It does not. Read train/grpo.py:153-169:

# Why no `env=` / `environment_url=` / `environment_factory` kwargs?
# - TRL 1.2.0's GRPOTrainer.__init__ only accepts env interaction via
#   `tools=` (needs transformers>=5.0), `environment_factory=`
#   (needs transformers>=5.2), or `rollout_func=`. We're on
#   transformers==4.56.2, so the first two raise. The third requires a
#   custom multi-turn rollout implementation we don't have time to
#   harden.
# - Our reward function (rewards.compose) operates on (prompt, completion,
#   gold_answer, prompt_token_count, llm_call_count) — zero env
#   interaction needed.

So:

• The env exists and is deployed (rubric requirement met).
• The env is used at inference time in the demo Space (judges can run a recursive episode).
• The env is NOT used at training time. GRPO is single-turn prompt → completion → deterministic reward.

A judge who reads code may flag this as inconsistent with the README's repeated claims about "teaching the model to use the REPL/recursion." That's a goodwill hit we can avoid with one paragraph of plain language.

D.2 Required README edit

In README.md, just after the Architecture section (around line 33), insert this paragraph verbatim:

### A note on the role of the env in training

TRL 1.2.0 with `transformers==4.56.2` does not yet expose multi-turn env-tool calls inside `GRPOTrainer.train()` (the `tools=` / `environment_factory=` kwargs require `transformers>=5.0`, and a custom `rollout_func=` was outside our time budget). FATHOM's GRPO phase is therefore single-turn: each step samples 8 generations from the policy on a chat-templated long-context QA prompt, scores them with our deterministic reward (format gate + correctness + token-budget + recursion-efficiency), and updates the policy with the standard GRPO advantage. **The env is exercised end-to-end at inference time** — the demo Space runs full multi-turn REPL + recursive `llm()` episodes against the trained model. Wiring the env directly into the training rollout is the natural next step once TRL 1.3 / transformers 5 ships.

This is honest, it preempts the obvious code-reading critique, and it actually reframes our submission as forward-looking rather than incomplete.

D.3 Architecture image touch-up (optional, only if time)

assets/architecture.png shows arrows from "GRPOTrainer" to "REPL" and "llm()". Either:

• (a) Edit the source assets/architecture.mmd (Mermaid) so those arrows are dashed and labeled inference-time only, or
• (b) Skip this if you've already done D.2 — the README paragraph carries enough context.

---

6. Execution Order (with checkpoints)

Step	Action	Time	Cost	Checkpoint
1	Read this brief, run git status, confirm clean working tree	5 min	$0	git status clean
2	Apply A.3.1 (prompt alignment) + A.3.2 (soft format) + A.3.3 (assert)	30 min	$0	CPU dry-run prints PREFIX_MATCH: True
3	Commit: fix(grpo): align prompt with SFT, soft format reward	5 min	$0	git log shows commit
4	Run smoke on HF Jobs a10g-large	5 min	~$0.10	outputs/smoke/SMOKE_RESULT.md GO
5	50-step GRPO trial run	20 min	~$2	reward_curve.png shows movement
6	Decision gate — full retrain or honest fallback	—	—	see A.4 step 3
7	Apply B.3 (env Space index) + push	15 min	$0	curl / returns HTML
8	Build B.4 (Streamlit demo Space) + push	60 min	$0	demo URL renders
9	Apply C.2 (mini-blog)	30 min	$0	blog Space live
10	Apply C.3 (GitHub mirror)	5 min	$0	GH repo public
11	Apply D.2 (README clarity paragraph)	5 min	$0	README diff
12	Run python scripts/submission_preflight.py	1 min	$0	"judge-ready" message
13	Refresh README submission table per C.4	10 min	$0	all rows filled
14	Final commit + push to HF master + GH main	5 min	$0	both remotes in sync
15	Hit every link in the README from a fresh browser	10 min	$0	nothing 404s

Total: ~3.5 h of work + ~$2–15 of cloud GPU depending on retrain decision.

---

7. Things You Might Be Tempted To Do — Don't

• ❌ Upgrade trl/transformers/unsloth to enable env-tool calls in training. This is a 2-day refactor with high failure risk. The D.2 paragraph defuses the critique without code changes.
• ❌ Switch reward composition from weighted-sum to product or RLHF-style ranking. The composition is fine; the format gate was the bug.
• ❌ Train a 3B model "for better optics." The CLAUDE.md is explicit that 1.5B was the deliberate choice. Sticking with 1.5B is part of the story (small-model recursion).
• ❌ Increase max_completion_length past 2048. STACK §10.3 calls this an anti-pattern. 2048 is fine.
• ❌ Move from vllm_mode='colocate' to 'server'. STACK §10.4 + TRL #4543 — this breaks multi-turn. We don't even use multi-turn in training, but colocate is also the cheaper option memory-wise.
• ❌ Refactor env/server/environment.py. It's stable, audited, and shipped. Touch nothing inside env/ except app.py for the index route.
• ❌ Delete the v1 flat-line GRPO plot. Honest evidence is part of the storytelling. Either replace with v2 (if A.4 succeeds) or annotate (if A.5 fallback).
• ❌ Generate fake Pareto numbers because the demo Space looks empty. Judges who notice are merciless. Either compute real numbers from data/eval.jsonl against the merged model or hide the column.
• ❌ Run pip install -U of anything inside the venue venv. The G10/G12 smoke gates passed on the current pin set; any upgrade voids that.

---

8. Reproduction Recipes (use when verifying)

8.1 Verify the flat-reward bug is real

# Convert the UTF-16 log Cursor wrote, then count how many steps had reward != 0
iconv -f UTF-16LE -t UTF-8 job9b_full.log 2>/dev/null \
  | grep -oE "'rewards/_instrumented_reward_fn/mean': [0-9.]+" \
  | sort -u
# Expected: only "'rewards/.../mean': 0.0" — confirms 100% flat

8.2 Verify the prompt-shape mismatch

python - <<'PY'
import json
sft = json.loads(open("data/sft_traces.jsonl", encoding="utf-8").readline())
print("SFT user content head:", repr(sft["messages"][1]["content"][:80]))
# Expected: starts with "Question: "

train = json.loads(open("data/train.jsonl", encoding="utf-8").readline())
# Simulate train/grpo.py's _to_prompt user content (PRE-FIX)
print("GRPO user content head (pre-fix):", repr(f"Context:\n{train['context'][:60]}\n\n{train['prompt']}"[:80]))
# Expected: starts with "Context:\n" — confirms the drift
PY

8.3 After A.3 fixes, dry-run the smoke test locally

python -m uvicorn env.server.app:app --host 0.0.0.0 --port 8001 &
sleep 5
python -m train.smoke_test --env-url http://localhost:8001

Expected: outputs/smoke/SMOKE_RESULT.md shows VERDICT: GO.

8.4 Verify Space deploy succeeded

for url in "/" "/healthz" "/state" "/docs"; do
  printf "GET $url → "
  curl -s -o /dev/null -w "%{http_code}\n" "https://Pratham-math-fathom-env.hf.space$url"
done
# Expected: 200, 200, 200, 200 — currently / returns 404.

---

9. Glossary (Antigravity is cold; here's the cheat sheet)

• OpenEnv — Meta's standard for RL environments. Defines the JSON contract /reset, /step, /state, /healthz. We pin openenv-core>=0.2.3,<0.3.
• GRPO — Group Relative Policy Optimization. The trainer samples N completions (here 8), computes per-group advantages (relative to group mean/std), and updates the policy. Critical: if all N completions get the same reward, std=0 → advantage=0 → no update. That's exactly our v1 failure mode.
• TRL — Hugging Face's RLHF/GRPO trainer library. We're on trl==1.2.0.
• Unsloth — Memory-efficient LoRA + 4-bit loader. We use Qwen2.5-Coder-1.5B-Instruct-bnb-4bit + LoRA r=16.
• RLM (Recursive Language Model) — a scaffold where an LM can call itself recursively on document chunks. We cap depth at 2 in training, 4 at demo.
• Format gate — the <answer>...</answer> regex check. Was multiplicative (binary 0/1), now should be additive (+0.10 bonus).
• vLLM colocate — vLLM runs in the same process as TRL trainer, sharing the GPU. Required for multi-turn (we don't use multi-turn in training but colocate is also cheaper memory-wise).

---

10. Definition of Done

You are done when all of these are true at once:

• python scripts/submission_preflight.py prints Preflight passed. Submission package looks judge-ready.
• curl -s -o /dev/null -w "%{http_code}" https://Pratham-math-fathom-env.hf.space/ returns 200 (not 404).
• https://Pratham-math-fathom-demo.hf.space renders a Streamlit page in <60 s with no placeholder/fake numbers visible.
• README's "Submission Links" table has zero _to be added_ placeholders.
• Mini-blog space is live and linked from README.
• GitHub mirror is live and linked from README.
• outputs/plots/grpo_reward.png either (a) shows non-zero variance after A.4 retrain, or (b) is honestly framed in README as v1 with diagnosis (per A.5 fallback).
• D.2 paragraph appears in README — env's training-vs-inference role is explicit.
• git status is clean on master; remote HF master + GitHub main are pushed and in sync.
• Smoke test green on HF Jobs (outputs/smoke/SMOKE_RESULT.md GO timestamp within last 24 h).

If any item is unchecked, you are not done. Do not declare victory and stop.

---

11. Hand-back Format

When you finish, append a single block to the bottom of this file:

## ANTIGRAVITY SESSION RESULT — <YYYY-MM-DD HH:MM IST>

- A (reward curve): <fixed and retrained / honestly reframed / blocked because X>
- B (Space UI): <env index + demo space deployed / partial / blocked>
- C (materials): <blog + GH mirror linked / partial / blocked>
- D (truth-in-advertising): <paragraph in / not in>
- Final preflight: <PASS / FAIL+reason>
- Cloud spend: $<x>
- Commits: <list of commit short-shas>

Open risks for the user before submission:
- ...

Stop after that block. Do not push the brief itself; the user committed it.