ANTIGRAVITY_BRIEF.md

# FATHOM — Antigravity Recovery Brief

**Project:** FATHOM (Meta × PyTorch × Hugging Face OpenEnv Hackathon Grand Finale, Bangalore, Apr 25–26 2026)
**Author of this brief:** triage handoff to Antigravity
**Today:** 2026-04-26 (final submission day)
**Repo root:** `C:\Users\prath\OneDrive\Desktop\Hackathons\Meta_finale\`
**Branch:** `master` (HF git remote is the source of truth; GitHub mirror not yet wired)
**HF user:** `Pratham-math`
**Live env Space:** <https://Pratham-math-fathom-env.hf.space> (CPU-basic, Docker SDK, RUNNING)
**Trained model repo:** <https://huggingface.co/Pratham-math/fathom-1.5b-grpo> (33 files, plots + adapters + merged_16bit live there)
**W&B run:** <https://wandb.ai/pratham-alwar05-indian-institute-of-information-technolo/huggingface/runs/sy1tqun0>

> **Recommended Antigravity model: Claude Sonnet 4.6** for bulk multi-file edits (deploy scripts, README, Streamlit, notebook, prompt-format alignment). **Escalate to Claude Opus 4.7** for the reward-design redesign in `rewards/compose.py` and the `train/grpo.py` prompt rewrite — those two need the hardest reasoning. **Gemini 2.5 Pro** is a fine substitute for either if you prefer Google models. Avoid running Haiku on this — the diagnosis below has too many interlocking pieces for a small model to keep coherent.

---

## 0. Read This First (Do Not Skip)

You are taking over a hackathon submission with **roughly 6–10 hours left** before judging. Code, training, and deployment are mostly built; **what's broken is small in surface area but huge in optics.** A judge clicking the README will see a flat-line reward curve, a JSON-only `/healthz` endpoint when they hit the Space URL, and a missing demo video link. Your job is to fix those three things in that order and stop. Do not refactor, do not rename, do not "clean up" — every change should be load-bearing for one of the four problems in §1.

**Hard rules for this session:**

1. **Do not retrain on a fresh A100 unless §2.A explicitly tells you to.** Cloud GPU minutes cost real money and the user has ~$10 of HF credits left. Every code change for §2.A must pass a **CPU dry-run** (no model weights, just shape + format checks) before you propose a re-train.
2. **Do not delete the existing failed run's evidence.** `outputs/plots/grpo_reward.png` is currently a flat line. Keep it, but reframe it in the README as a "v1, diagnosed" curve, then publish a v2 next to it once the fix lands. Honest evidence > deleted evidence.
3. **Stay on the existing stack.** No `pip install -U` of `unsloth`, `trl`, `transformers`, `vllm`, `peft`, or `bitsandbytes`. The pinned versions in `pyproject.toml` are the ones the smoke test passed on. If a library upgrade tempts you, the answer is no.
4. **Commit at every milestone with conventional-commit prefixes** (`fix(grpo):`, `feat(space):`, `docs(readme):`). The user reviews PRs by reading commits, not diffs.
5. **All HF pushes go to existing repos** (`Pratham-math/fathom-env`, `Pratham-math/fathom-1.5b-grpo`, `Pratham-math/fathom-code`). Do not create new HF repos.
6. **If you cannot reproduce a problem, say so.** Do not "fix" things by guessing. Every fix in §2 has a reproduction script you can run.

---

## 1. The Four Problems, Ranked

| # | Problem | Severity | Time to fix | Judges-visible? |
|---|---------|----------|-------------|----------------|
| **A** | GRPO reward curve is a flat line at 0.0 for every step | **CRITICAL** | 2–3 h (incl. retrain) | Yes — README plot |
| **B** | HF Space has no UI; judges see `{"detail":"Not Found"}` at root | **HIGH** | 1.5–2 h | Yes — first impression |
| **C** | Missing materials: mini-blog, video, slides, GitHub mirror | **HIGH** | 1.5 h | Yes — non-negotiable rubric items |
| **D** | The README implies the model "uses recursion in training"; it doesn't | **MEDIUM** | 30 min | Yes — judges may grep |

---

## 2. Problem A — Flat Reward Curve

### A.1 What the user sees

`outputs/plots/grpo_reward.png` (and the same file at <https://huggingface.co/Pratham-math/fathom-1.5b-grpo/blob/main/plots/grpo_reward.png>) is a horizontal line at y=0.0 across all logged GRPO steps. The reward never moves. Even though SFT loss looks healthy (3.2 → 0.29) and SFT token accuracy hits 0.93, the GRPO phase teaches the model nothing.

### A.2 What is actually happening (root cause, verified)

I extracted training metrics from `job9b_full.log` (UTF-16-encoded; convert with `iconv -f UTF-16LE -t UTF-8`). **Every** logged GRPO step looks like this:

```python
{'loss': 0.0,
 'completions/mean_length': 3.25,        # ← model outputs ~3 tokens like "the man."
 'completions/min_length': 3.0,
 'completions/max_length': 4.0,
 'rewards/_instrumented_reward_fn/mean': 0.0,   # ← every generation scores 0
 'reward_std': 0.0,                              # ← all 8 generations identical reward
 'frac_reward_zero_std': 1.0,                    # ← GRPO advantage is 0 for 100% of examples
 'kl': 0.0,
 'clip_ratio/region_mean': 0.0,
 ...}
```

The chain of failure is:

1. The SFT-warm-started Qwen-1.5B is fine-tuned on `data/sft_traces.jsonl`, where each user message is shaped:

   ```
   Question: <q>

   [Document excerpt]:
   <ctx>
   ```

   The assistant target ends with `<answer>...</answer>`.

2. GRPO loads that SFT adapter, then `train/grpo.py:218-230` builds an entirely **different** user message shape:

   ```
   Context:
   <ctx_truncated>

   <example.prompt>
   ```

   Note the order is reversed (`Context:` first vs. `Question:` first), the field names differ (`Context:` vs. `[Document excerpt]:`), and the SFT-trained model has never seen this layout.

3. Confronted with an out-of-distribution prompt, the model collapses to the lowest-loss continuation it knows: bare 2–13 token answer spans like `the man.` or `silver.` — never wrapped in `<answer>...</answer>`.

4. `rewards/format_gate.py` is a **multiplicative** gate. Missing `<answer>` tag → `format_gate=0.0` → `compose.py:32` short-circuits the entire composite to `0.0`.

5. All 8 GRPO generations score exactly 0.0. **GRPO advantage = (reward − group mean) / group std = 0 / 0 → 0.** Gradient is therefore 0. The policy never moves. **Straight line forever.**

This is a single diagnosis with two compounding causes: **(i) prompt-shape drift between SFT and GRPO** and **(ii) a multiplicative gate with no soft floor**. Either one alone would degrade learning; together they zero it out. The recent commits (`fix(grpo): align sys_msg with SFT`, `flip Path 3 -> Path 1`) addressed the system message but not the user-content shape, and not the gate.

### A.3 Required fixes (apply all three; they are not redundant)

**Fix A.3.1 — Align the GRPO user-message shape with SFT.** Edit `train/grpo.py`:

```python
# train/grpo.py — replace lines 218-239 (the _to_prompt function)

def _to_prompt(example: dict) -> dict:
    ctx_full = example.get("context", "") or ""
    ctx_truncated = _truncate_to_tokens(ctx_full, ctx_budget_tok)
    # CRITICAL: must match data/sft_traces.jsonl user-message shape exactly.
    # SFT used "Question: <q>\n\n[Document excerpt]:\n<ctx>". Any deviation
    # puts the SFT-warm-started policy out-of-distribution and collapses
    # generation length to ~3 tokens (verified job9b_full.log).
    user_content = (
        f"{example.get('prompt', '')}\n\n"
        f"[Document excerpt]:\n{ctx_truncated}"
    )
    msgs = [
        {"role": "system", "content": sys_msg},
        {"role": "user", "content": user_content},
    ]
    prompt_str = tokenizer.apply_chat_template(
        msgs, tokenize=False, add_generation_prompt=True
    )
    return {
        "prompt": prompt_str,
        "gold_answer": str(example.get("gold_answer", "")),
        "prompt_token_count": int(example.get("context_length", 0)) // 4,
        "llm_call_count": 0,
    }
```

Also replace the `sys_msg` string (currently lines 187–192) with the **exact** system message that appears in `data/sft_traces.jsonl`, which is:

```
You are FATHOM, a recursive language model with a Python REPL sandbox. You can read a long document via the variable `ctx` and call `llm(prompt, chunk)` for sub-queries. Think step by step. Emit your final answer inside <answer>...</answer>.
```

(You can grep this from the first line of `data/sft_traces.jsonl` to confirm.) **Do not paraphrase.** Byte-identical or the SFT adapter will not transfer.

**Verification of A.3.1 (CPU-only, no GPU):**

```bash
python - <<'PY'
import json
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-1.5B-Instruct")
sft = json.loads(open("data/sft_traces.jsonl", encoding="utf-8").readline())
sft_msgs = sft["messages"][:2]  # system + user
sft_str = tok.apply_chat_template(sft_msgs, tokenize=False, add_generation_prompt=True)

# Now build the GRPO-side equivalent from data/train.jsonl row 0
row = json.loads(open("data/train.jsonl", encoding="utf-8").readline())
sys_msg = sft_msgs[0]["content"]
user = f"{row['prompt']}\n\n[Document excerpt]:\n{row['context'][:2000]}"
grpo_msgs = [{"role":"system","content":sys_msg},{"role":"user","content":user}]
grpo_str = tok.apply_chat_template(grpo_msgs, tokenize=False, add_generation_prompt=True)
# The system+user prefix should match byte-for-byte up to where the contexts differ
print("PREFIX_MATCH:", sft_str[:400] == grpo_str[:400])
PY
```

You should see `PREFIX_MATCH: True`. If False, the system message or chat template formatting still differs — keep iterating until True.

**Fix A.3.2 — Soft-format reward instead of a binary gate.** Edit `rewards/compose.py`. Replace `compose_reward_single` body so that missing `<answer>` no longer zeroes the composite; instead it loses a 0.10 bonus and gets bounded above by 0.05 (existing A-02 cap remains). This gives GRPO a non-zero gradient even when the policy is initially mis-formatted.

```python
# rewards/compose.py — replace compose_reward_single (lines 19-63)
def compose_reward_single(
    completion: str,
    gold_answer: str,
    prompt_token_count: int,
    llm_call_count: int,
    cfg_reward: Any,
) -> float:
    """Composite reward — soft format bonus instead of multiplicative gate.

    REW-02 v2: GRPO collapses when a multiplicative gate yields std=0 across
    a group (every generation scores 0). Replace with an additive 0.10
    format bonus so even malformed generations carry signal, then keep the
    A-02 correctness==0 cap at 0.05 to block format-only exploits.
    """
    has_format = format_gate(completion) == 1.0
    c = correctness(completion, gold_answer) if has_format else 0.0
    t = token_budget(
        completion,
        prompt_token_count,
        alpha=float(cfg_reward.alpha),
        variant=str(cfg_reward.token_budget_variant),
    )
    r = recursion_efficiency(
        int(llm_call_count),
        max_calls=int(cfg_reward.get("max_calls", 2))
        if hasattr(cfg_reward, "get")
        else int(getattr(cfg_reward, "max_calls", 2)),
    )
    w = cfg_reward.weights
    assert abs(float(w.correctness) + float(w.token_budget) + float(w.recursion_efficiency) - 1.0) < 1e-3

    composite = (
        float(w.correctness) * c
        + float(w.token_budget) * t
        + float(w.recursion_efficiency) * r
    )
    # NEW: small additive format bonus — the only signal when the model is
    # still learning the template. Keeps GRPO advantages non-zero.
    if has_format:
        composite += 0.10

    # Anti-hacking caps:
    # 1. correctness==0 → at most 0.05 (blocks format-only exploit)
    if c == 0.0:
        return min(composite, 0.05 + (0.10 if has_format else 0.0))
    return composite
```

Update `REWARD_AUDIT.md` so the A-01 row reflects the new ceiling: `<answer></answer>` (empty answer with format) now scores at most **0.10** (the bonus alone, since correctness=0 and the cap is `0.05 + 0.10 = 0.15`; refine your cap math accordingly). Re-run `pytest tests/test_rewards.py -k "audit"` and confirm everything still passes; fix the assertions in the audit tests if their expected values shift.

**Fix A.3.3 — Add a regex-driven assertion before `trainer.train()` runs.** This is your insurance against the bug coming back silently. In `train/grpo.py`, just before `trainer.train()`:

```python
# Pre-flight: tokenize one example and confirm the chat-template prefix is
# the byte-identical match of an SFT trace prefix. If not, the SFT adapter
# is loaded but the policy will be out-of-distribution and reward will
# collapse (root cause of the v1 flat-line run).
import json as _json
_sft = _json.loads(open(str(cfg.data.sft_traces_path) if hasattr(cfg.data, "sft_traces_path") else "data/sft_traces.jsonl", encoding="utf-8").readline())
_sft_prefix = tokenizer.apply_chat_template(_sft["messages"][:2], tokenize=False, add_generation_prompt=True)[:200]
_grpo_first = train_dataset[0]["prompt"][:200]
assert _sft_prefix.split("Question:")[0] == _grpo_first.split("Question:")[0], (
    "SFT/GRPO chat-template prefix drift detected — see ANTIGRAVITY_BRIEF.md §A.3.1"
)
```

The exact split key may need tweaking depending on the tokenizer output; the goal is "if the system block diverges, raise loudly."

---

### A.4-bis — Real recursion-efficiency signal (the "actually teach the model to plan recursion" patch)

**Why this exists.** With only A.3.1–A.3.3, GRPO will start moving but it's optimizing for "produce a correctly-formatted exact-match answer." It is not optimizing for *when to recurse vs. when not to*. The current `recursion_efficiency` reward is dead — `train/grpo.py:239` hardcodes `llm_call_count=0` for every example, so that 5% weight is constant across all 8 generations and contributes zero variance. We're going to wake it up by:

1. **Parsing the model's completion** for `llm(` calls inside fenced Python code blocks
2. **Coupling the efficiency bonus to correctness** so the model can't farm reward by emitting `llm(` strings without solving the task
3. **Rebalancing weights** to give recursion behavior a real say (0.05 → 0.15)

This converts FATHOM's GRPO from "single-turn QA training" to "plan-grading training." The model still doesn't *execute* recursion during the rollout (that requires the §D rewrite which is out of budget), but it learns to **predict good plans**: which task types deserve `llm()` calls and which don't. The trained model then drops into the inference-time recursion scaffold and executes those plans for real.

#### A.4-bis.1 The dataset — what the reward is actually shaping behavior across

`data/train.jsonl` contains 1000 rows with this distribution (verified):

| Task type | Count | Example prompt | Optimal recursion |
|-----------|-------|----------------|-------------------|
| `niah` | 400 (40%) | "What color is the mirror?" | **0 calls** — REPL grep finds the needle |
| `extractive` | 200 (20%) | "In which city was the 2000 agreement signed?" | **0 calls** — REPL regex |
| `multi_needle` | 300 (30%) | "Total cost of apple, lemon, plum?" | **0–2 calls** — REPL extracts; `llm()` if chunk too large |
| `counting` | 100 (10%) | "How many times does 'apple' appear?" | **0 calls** — REPL only (LLMs are bad at counting) |

Context lengths: min 4K, median 16K, p90 200K, max 200K. Larger contexts increasingly need `llm()` calls because they get tail-truncated to 4K in the prompt.

The reward function does **NOT** see `task_type`. The model has to *infer* recursion need from the prompt structure and context length — that's the right kind of generalization to teach.

#### A.4-bis.2 Create `rewards/recursion_extract.py` (NEW FILE)

Extracts `llm(` call counts from completion text, ignoring strings/comments using Python's `tokenize` module (with a regex fallback for syntactically-invalid blocks).

```python
"""Recursion call extractor — REW-04 v2.

Counts llm( function calls inside fenced ```python code blocks of a model
completion. Ignores occurrences in:
  - prose outside any code block
  - comments inside a code block (# llm(...) → 0)
  - string literals inside a code block ("did llm(...)" → 0)

Uses Python's tokenize module for accuracy; regex fallback when the code
block is syntactically invalid (the model writes broken Python sometimes
but we still want to count its intent).
"""
from __future__ import annotations

import io
import re
import tokenize

# Match ``` or ```python or ```py — case-insensitive, multi-line.
_CODE_BLOCK_RE = re.compile(
    r"```(?:python|py)?\s*\n(.*?)```",
    re.DOTALL | re.IGNORECASE,
)
_LLM_CALL_RE = re.compile(r"\bllm\s*\(")


def _count_in_block(code: str) -> int:
    """Count llm( calls in one code block. Tokenize-aware; regex fallback."""
    try:
        toks = list(tokenize.generate_tokens(io.StringIO(code).readline))
    except (tokenize.TokenizeError, IndentationError, SyntaxError):
        # Strip line comments, then regex. Conservative — does not strip
        # string literals, but the model rarely puts llm( in a string when
        # writing broken code.
        stripped = "\n".join(line.split("#", 1)[0] for line in code.splitlines())
        return len(_LLM_CALL_RE.findall(stripped))

    count = 0
    for i in range(len(toks) - 1):
        tok = toks[i]
        nxt = toks[i + 1]
        if (
            tok.type == tokenize.NAME
            and tok.string == "llm"
            and nxt.type == tokenize.OP
            and nxt.string == "("
        ):
            count += 1
    return count


def count_llm_calls(completion: str) -> int:
    """Total llm( calls inside all fenced code blocks of the completion.

    Returns 0 if completion is empty, has no code blocks, or only contains
    llm( in prose / comments / strings.
    """
    if not completion:
        return 0
    blocks = _CODE_BLOCK_RE.findall(completion)
    if not blocks:
        return 0
    return sum(_count_in_block(b) for b in blocks)


__all__ = ["count_llm_calls"]
```

#### A.4-bis.3 Replace `rewards/recursion_efficiency.py`

```python
"""Recursion efficiency reward component — REW-04 v2.

Linear decay on llm_call_count. Pure Python, stdlib-only.

Intended ranges:
  0 calls  → 1.0 (best — task didn't need recursion)
  1 call   → 0.75
  2 calls  → 0.50
  3 calls  → 0.25
  4+ calls → 0.00 (recursion spam is wasteful)

This score is *coupled to correctness* in compose.py — wrong answers don't
earn an efficiency bonus, which prevents the model from learning to spam
`llm(` strings in code blocks for free reward.
"""

def recursion_efficiency(llm_call_count: int, **_) -> float:
    """Linear-decay efficiency on call count; gated to correctness in compose."""
    count = max(0, int(llm_call_count))
    return max(0.0, 1.0 - 0.25 * count)


__all__ = ["recursion_efficiency"]
```

#### A.4-bis.4 Replace `rewards/compose.py`

```python
"""Reward composition — REW-02 v3.

Changes from v2 (the §A.3.2 "soft format bonus" patch):
  - llm_call_count is now extracted from the completion's fenced Python
    code blocks (via rewards.recursion_extract.count_llm_calls), not
    hardcoded to 0 in train/grpo.py.
  - Recursion efficiency is gated on correctness — wrong answers cannot
    earn an efficiency bonus. Prevents the model from spamming `llm(`
    strings to harvest free reward.
  - Weights rebalanced: 0.70 correctness / 0.15 token_budget /
    0.15 recursion_efficiency. (Was 0.75 / 0.20 / 0.05.)
  - Per-component scalars are returned alongside the composite via the
    `_metrics` dict so the GRPOTrainer wrapper in train/grpo.py can
    log real per-component means to W&B (currently logs 0.0).

Anti-hacking caps preserved:
  - c == 0.0 → composite ≤ 0.25 (was 0.05; raised to allow soft-format
    bonus to register, still well below any correct answer ≥ 0.80).
"""
from __future__ import annotations

from typing import Any, Callable

from .format_gate import format_gate
from .correctness import correctness
from .token_budget import token_budget
from .recursion_efficiency import recursion_efficiency
from .recursion_extract import count_llm_calls


def compose_reward_single(
    completion: str,
    gold_answer: str,
    prompt_token_count: int,
    cfg_reward: Any,
    llm_call_count: int | None = None,  # if None → extract from completion
) -> tuple[float, dict[str, float]]:
    """Single-example composite reward + per-component metrics.

    Returns (composite_score, metrics_dict). The metrics dict has keys:
        format_pass, correctness, token_budget, recursion_eff_raw,
        recursion_eff_contribution, llm_call_count.
    """
    has_format = format_gate(completion) == 1.0
    c = correctness(completion, gold_answer) if has_format else 0.0
    t = token_budget(
        completion,
        prompt_token_count,
        alpha=float(cfg_reward.alpha),
        variant=str(cfg_reward.token_budget_variant),
    )

    if llm_call_count is None:
        llm_call_count = count_llm_calls(completion)
    eff_raw = recursion_efficiency(int(llm_call_count))
    # Couple efficiency to correctness — wrong answers earn 0 efficiency.
    eff_contribution = eff_raw if c == 1.0 else 0.0

    w = cfg_reward.weights
    assert abs(
        float(w.correctness) + float(w.token_budget) + float(w.recursion_efficiency) - 1.0
    ) < 1e-3, "REW-02 v3: composite weights must sum to 1.0"

    composite = (
        float(w.correctness) * c
        + float(w.token_budget) * t
        + float(w.recursion_efficiency) * eff_contribution
    )
    if has_format:
        composite += 0.10  # soft format bonus (§A.3.2)

    if c == 0.0:
        composite = min(composite, 0.25)

    metrics = {
        "format_pass": 1.0 if has_format else 0.0,
        "correctness": c,
        "token_budget": t,
        "recursion_eff_raw": eff_raw,
        "recursion_eff_contribution": eff_contribution,
        "llm_call_count": float(llm_call_count),
    }
    return composite, metrics


def compose_reward_fn(prompts: list, completions: list, **kwargs) -> list[float]:
    """TRL-compatible batched reward function. Returns scalars only.

    Per-component means are stashed under `kwargs['_component_means']` for
    the GRPOTrainer instrumentation wrapper to log to W&B. (TRL ignores
    extra kwargs.)
    """
    cfg_reward = kwargs.pop("cfg_reward")
    gold_answers = kwargs.get("gold_answer", [""] * len(completions))
    ptcs = kwargs.get("prompt_token_count", [1] * len(completions))

    pairs = [
        compose_reward_single(c, g, int(p), cfg_reward)
        for c, g, p in zip(completions, gold_answers, ptcs)
    ]
    rewards = [p[0] for p in pairs]
    metrics_list = [p[1] for p in pairs]

    # Aggregate component means for W&B logging via the wrapper.
    if metrics_list:
        keys = metrics_list[0].keys()
        means = {k: sum(m[k] for m in metrics_list) / len(metrics_list) for k in keys}
        kwargs["_component_means"] = means
    return rewards


def make_reward_fn(cfg_reward: Any) -> Callable:
    """Factory binding cfg_reward for GRPOTrainer.reward_funcs."""
    def _bound(prompts, completions, **kwargs):
        kwargs["cfg_reward"] = cfg_reward
        return compose_reward_fn(prompts, completions, **kwargs)
    return _bound


__all__ = ["compose_reward_fn", "compose_reward_single", "make_reward_fn"]
```

#### A.4-bis.5 Patch `train/grpo.py`

Two edits:

**(a)** In `_to_prompt` (lines 218–239 in current file, will shift after §A.3.1), **delete** the `"llm_call_count": 0` field from the returned dict — the extractor now computes it from each rollout's completion. Final return shape:

```python
return {
    "prompt": prompt_str,
    "gold_answer": str(example.get("gold_answer", "")),
    "prompt_token_count": int(example.get("context_length", 0)) // 4,
}
```

**(b)** Replace the `_instrumented_reward_fn` body (currently at lines 138–151) so it logs the **real** per-component means that `compose_reward_fn` now stashes under `kwargs['_component_means']`:

```python
def _instrumented_reward_fn(prompts, completions, **kwargs):
    rewards = reward_fn(prompts, completions, **kwargs)
    try:
        if wandb.run is not None:
            log_dict = {
                "reward/composite_mean": sum(rewards) / max(len(rewards), 1),
                "reward/composite_std": (
                    statistics.stdev(rewards) if len(rewards) > 1 else 0.0
                ),
            }
            cm = kwargs.get("_component_means", {})
            for k, v in cm.items():
                log_dict[f"reward/{k}_mean"] = float(v)
            wandb.log(log_dict)
    except Exception:
        pass
    return rewards
```

Add `import statistics` at the top of the file (already has `import inspect`, `import logging`, `import os`, etc., so just add the line).

#### A.4-bis.6 Update `configs/reward/v1.yaml`

```yaml
alpha: 0.2
weights:
  correctness: 0.70
  token_budget: 0.15
  recursion_efficiency: 0.15
token_budget_variant: "capped_linear"
answer_regex: "<answer>(.*?)</answer>"
max_calls: 4
```

#### A.4-bis.7 Add tests (`tests/test_recursion_extract.py`, NEW)

```python
"""Tests for rewards.recursion_extract — REW-04 v2."""
from rewards.recursion_extract import count_llm_calls


def test_empty_completion():
    assert count_llm_calls("") == 0


def test_no_code_block():
    assert count_llm_calls("The answer is <answer>silver</answer>") == 0


def test_single_call():
    c = "```python\nresult = llm('find', ctx[:5000])\n```\n<answer>silver</answer>"
    assert count_llm_calls(c) == 1


def test_two_calls_in_one_block():
    c = "```python\na = llm('q1', ctx[:1000])\nb = llm('q2', ctx[1000:])\n```"
    assert count_llm_calls(c) == 2


def test_calls_across_two_blocks():
    c = "```python\nx=llm('q', ctx)\n```\nthen\n```python\ny=llm('q2', ctx)\n```"
    assert count_llm_calls(c) == 2


def test_call_in_comment_not_counted():
    c = "```python\n# would call llm(stuff) but skipping\nprint('done')\n```"
    assert count_llm_calls(c) == 0


def test_call_in_string_literal_not_counted():
    c = '```python\nnote = "earlier code did llm(...)"\nprint(note)\n```'
    assert count_llm_calls(c) == 0


def test_call_outside_code_block_not_counted():
    c = "Maybe I should call llm(question, chunk) but I won't actually."
    assert count_llm_calls(c) == 0


def test_call_in_loop_counts_literal_occurrence():
    c = "```python\nfor chunk in chunks:\n    r = llm('find', chunk)\n```"
    assert count_llm_calls(c) == 1


def test_invalid_python_falls_back_to_regex():
    c = "```python\nthis is not valid python !!!\nresult = llm('q', ctx)\n```"
    assert count_llm_calls(c) >= 1  # fallback regex finds it


def test_bare_python_fence():
    c = "```\nans = llm('q', ctx)\n```"  # no language tag
    assert count_llm_calls(c) == 1
```

#### A.4-bis.8 Add tests to `tests/test_rewards.py`

Append a new test class at the end:

```python
class TestComposeV3:
    """REW-02 v3: soft format + recursion-extraction + correctness-gated efficiency."""

    @pytest.fixture
    def cfg_v3(self):
        return OmegaConf.create({
            "alpha": 0.2,
            "weights": {"correctness": 0.70, "token_budget": 0.15, "recursion_efficiency": 0.15},
            "token_budget_variant": "capped_linear",
            "answer_regex": "<answer>(.*?)</answer>",
            "max_calls": 4,
        })

    def test_correct_no_recursion_scores_high(self, cfg_v3):
        c = "```python\nimport re\nm=re.search('silver', ctx)\nprint(m.group())\n```\n<answer>silver</answer>"
        score, metrics = compose_reward_single(c, "silver", 100, cfg_v3)
        assert score >= 0.85, f"clean correct should score high, got {score}"
        assert metrics["llm_call_count"] == 0

    def test_zero_calls_beats_one_call_when_both_correct(self, cfg_v3):
        c0 = "```python\nimport re\nm=re.search('silver', ctx)\nprint(m.group())\n```\n<answer>silver</answer>"
        c1 = "```python\nans=llm('color', ctx[:5000])\nprint(ans)\n```\n<answer>silver</answer>"
        s0, _ = compose_reward_single(c0, "silver", 100, cfg_v3)
        s1, _ = compose_reward_single(c1, "silver", 100, cfg_v3)
        assert s0 > s1, f"0-call ({s0:.3f}) should beat 1-call ({s1:.3f}) when both correct"

    def test_efficiency_gated_on_correctness(self, cfg_v3):
        # Wrong answer with 0 calls — must NOT earn efficiency bonus.
        c = "```python\nprint('done')\n```\n<answer>gold</answer>"
        score, metrics = compose_reward_single(c, "silver", 100, cfg_v3)
        assert metrics["recursion_eff_contribution"] == 0.0
        assert score <= 0.25, f"wrong answer must be capped, got {score}"

    def test_recursion_spam_loses_to_minimal_recursion(self, cfg_v3):
        c2 = "```python\na=llm('q1',ctx[:1000])\nb=llm('q2',ctx[1000:2000])\n```\n<answer>silver</answer>"
        c5 = "```python\n" + "\n".join(f"x{i}=llm('q{i}',ctx)" for i in range(5)) + "\n```\n<answer>silver</answer>"
        s2, _ = compose_reward_single(c2, "silver", 100, cfg_v3)
        s5, _ = compose_reward_single(c5, "silver", 100, cfg_v3)
        assert s2 > s5, f"2-call ({s2:.3f}) should beat 5-call spam ({s5:.3f})"

    def test_format_only_capped(self, cfg_v3):
        c = "<answer>wrong</answer>"
        score, _ = compose_reward_single(c, "silver", 100, cfg_v3)
        assert 0.05 <= score <= 0.25, f"format-only wrong should be in [0.05, 0.25], got {score}"

    def test_no_format_gets_minimal_credit(self, cfg_v3):
        c = "silver"  # right text but no <answer> tag
        score, _ = compose_reward_single(c, "silver", 100, cfg_v3)
        assert score <= 0.20

    def test_group_variance_nonzero(self, cfg_v3):
        """Smoke check: a synthetic GRPO group of 8 must produce non-zero std.
        v1 had std=0.0 across all groups, which zeroed the GRPO advantage."""
        gens = [
            "```python\nimport re\nm=re.search('silver',ctx)\nprint(m.group())\n```\n<answer>silver</answer>",
            "```python\nans=llm('color',ctx[:5000])\nprint(ans)\n```\n<answer>silver</answer>",
            "<answer>silver</answer>",
            "<answer>gold</answer>",
            "the color is silver",
            "<answer></answer>",
            "```python\n" + "\n".join(f"x{i}=llm('q{i}',ctx)" for i in range(5)) + "\n```\n<answer>silver</answer>",
            "silver.",
        ]
        scores = [compose_reward_single(g, "silver", 200, cfg_v3)[0] for g in gens]
        import statistics as _st
        assert _st.stdev(scores) > 0.10, f"group std too low: {_st.stdev(scores)}"
```

Note: existing tests in `TestComposeReward` will break because `compose_reward_single` now returns `(score, metrics)` instead of just `score`. Either:
- (a) Update the existing tests to unpack `(score, _) = compose_reward_single(...)`, or
- (b) Keep backward compat by adding a `return_metrics: bool = False` flag with default False that returns just the float.

**Pick (a)** — explicit is better, and the v1 tests' expected values change anyway under the new weights. Find any existing call site of `compose_reward_single` and add `, _` to the unpacking. Update the old `TestComposeReward` cases to use new expected ranges (the cap moved from 0.05 to 0.25).

#### A.4-bis.9 Update `REWARD_AUDIT.md`

Add two new attack rows and revise A-05:

```markdown
## A-05: Recursion depth gaming (REVISED for v3)

**Vector:** Model uses 0 llm() calls on every task to maximize
recursion_efficiency, even on multi_needle / 200K tasks where recursion
would actually help correctness.

**Analysis (v3):**
- recursion_efficiency contributes only when correctness == 1.0 (gating
  in compose.py). On hard tasks where 0 calls fails to produce a correct
  answer, the efficiency bonus is forfeited entirely.
- Net incentive: use the *minimum* recursion that still produces a
  correct answer. Exactly the desired behavior.

**Status:** ✅ MITIGATED by correctness-gating.

---

## A-07: Comment-spam exploit (NEW)

**Vector:** Model emits `# llm(foo)` inside code blocks to inflate the
count regex without making real calls. (Inverted variant of A-05: spam
to make recursion_eff *lower*, useless because lower efficiency hurts.)

**Test:** `test_call_in_comment_not_counted` in
`tests/test_recursion_extract.py`.

**Result:** 0 calls counted ✅ — extractor uses tokenize, ignores comments.

**Status:** ✅ MITIGATED by tokenize-aware extraction.

---

## A-08: String-literal exploit (NEW)

**Vector:** Model writes `"earlier code did llm(...)"` in a string
literal to confuse a naive regex extractor.

**Test:** `test_call_in_string_literal_not_counted`.

**Result:** 0 calls counted ✅ — tokenize correctly identifies STRING
tokens and skips them.

**Status:** ✅ MITIGATED.
```

Update the bottom summary table accordingly. Bump verdict timestamp to today.

#### A.4-bis.10 CPU-only verification script (`scripts/verify_recursion_reward.py`, NEW)

This is the gate that must pass before any HF Job spend. Runs in <1 s on a laptop.

```python
"""CPU-only verification of REW-04 v2 reward design.

Confirms two GRPO-blocking properties:
  1. A synthetic 8-completion group produces non-zero std (v1's std was 0.0
     across every group, which is why the reward curve was flat).
  2. The ordering correct+0calls > correct+1call > correct+spam holds.

Run BEFORE spending any HF Jobs credits on a retrain.
"""
from __future__ import annotations

import statistics
import types

from rewards.compose import compose_reward_single

cfg = types.SimpleNamespace(
    alpha=0.2,
    weights=types.SimpleNamespace(
        correctness=0.70, token_budget=0.15, recursion_efficiency=0.15
    ),
    token_budget_variant="capped_linear",
    answer_regex="<answer>(.*?)</answer>",
    max_calls=4,
)

GOLD = "silver"
GENERATIONS = [
    ("correct + 0 llm calls (REPL grep)",
     "```python\nimport re\nm=re.search('silver', ctx)\nprint(m.group())\n```\n<answer>silver</answer>"),
    ("correct + 1 llm call",
     "```python\nans=llm('color', ctx[:5000])\nprint(ans)\n```\n<answer>silver</answer>"),
    ("correct + 3 llm calls (wasteful)",
     "```python\na=llm('q1',ctx[:1000])\nb=llm('q2',ctx[1000:2000])\nc=llm('q3',ctx[2000:3000])\n```\n<answer>silver</answer>"),
    ("correct + bare answer (no code, trivial-task path)",
     "<answer>silver</answer>"),
    ("wrong + format",
     "<answer>gold</answer>"),
    ("wrong + no format",
     "the color is gold"),
    ("right text + no format (v1 collapse mode)",
     "silver"),
    ("format-only spam",
     "<answer></answer>"),
]

print(f"{'idx':>3}  {'score':>6}  {'calls':>5}  description")
print("-" * 78)
scores = []
for i, (desc, gen) in enumerate(GENERATIONS):
    s, m = compose_reward_single(gen, GOLD, 200, cfg)
    scores.append(s)
    print(f"{i:>3}  {s:>6.3f}  {int(m['llm_call_count']):>5d}  {desc}")
print("-" * 78)
print(f"group mean: {statistics.mean(scores):.4f}")
print(f"group std:  {statistics.stdev(scores):.4f}  (must be > 0.10 for GRPO advantage)")
print(f"max - min:  {max(scores) - min(scores):.4f}")

# Hard gates — exit non-zero if any fail
assert statistics.stdev(scores) > 0.10, "FAIL: group std too low; GRPO will not learn"
assert scores[0] > scores[1] > scores[2], (
    f"FAIL: efficiency ordering broken (got {scores[0]:.3f} > {scores[1]:.3f} > {scores[2]:.3f})"
)
assert scores[0] > scores[4], "FAIL: correct must beat wrong"
assert scores[7] <= 0.25, "FAIL: format-only spam not capped"
print("\nPASS: REW-04 v2 produces learnable variance and correct orderings")
```

#### A.4-bis.11 Execution order — drop-in replacement for §6 steps 2–6

Replace steps 2–6 in the §6 table with:

| Step | Action | Time | Cost | Checkpoint |
|------|--------|------|------|------------|
| 2a | Apply A.3.1 (prompt alignment) + A.3.3 (assert) | 20 min | $0 | CPU dry-run prints `PREFIX_MATCH: True` |
| 2b | Apply A.4-bis: create `recursion_extract.py`, replace `recursion_efficiency.py`, replace `compose.py`, patch `train/grpo.py`, update `configs/reward/v1.yaml` | 40 min | $0 | All edits made, no test imports broken |
| 2c | Run new tests | 5 min | $0 | `pytest tests/test_recursion_extract.py tests/test_rewards.py -q` all green |
| 2d | Run CPU verifier | 1 min | $0 | `python scripts/verify_recursion_reward.py` prints `PASS:` |
| 3 | Commit: `fix(reward): align prompt with SFT, soft format, real recursion signal (A.3 + A.4-bis)` | 5 min | $0 | git log shows commit |
| 4 | Smoke on HF Jobs `a10g-large` | 5 min | ~$0.10 | `outputs/smoke/SMOKE_RESULT.md` GO |
| 5 | 50-step GRPO trial | 20 min | ~$2 | reward curve shows movement, group std > 0 |
| 6 | Decision gate (full retrain or honest fallback) | — | — | see A.4 below |

#### A.4-bis.12 Definition of done for this addendum

- [ ] `pytest tests/test_recursion_extract.py -q` → 11 passed
- [ ] `pytest tests/test_rewards.py -q` → all green (existing + new TestComposeV3 class)
- [ ] `python scripts/verify_recursion_reward.py` → exits 0 with `PASS:` line
- [ ] On the 50-step trial run, W&B shows non-zero values for `reward/correctness_mean`, `reward/recursion_eff_contribution_mean`, `reward/llm_call_count_mean` — not just `reward/composite_mean`
- [ ] `frac_reward_zero_std` in the trainer logs is `< 0.5` for at least 80% of steps (v1 was `1.0` for 100% of steps)

If item 5 above fails (frac_reward_zero_std stays at 1.0), the prompt-alignment fix in A.3.1 didn't take. Re-check `PREFIX_MATCH: True` and that the SFT adapter is actually loading (look for the log line `TRN-03 SFT adapter loaded from .../sft_adapter (after unloading empty wrap)`).

---

### A.4 Re-train and republish

Once A.3.1–A.3.3 land and CPU dry-run prints `PREFIX_MATCH: True`:

1. **Smoke test on HF Jobs first** (1 min, ~$0.10 on `a10g-large`):

   ```bash
   bash scripts/job_smoke.sh
   ```

   Inspect `outputs/smoke/SMOKE_RESULT.md`. Expected: GO with 6/6 PASS.

2. **Short GRPO run** — 50 steps only, NOT 400. This is to verify the curve moves. Use `a10g-large` (≈$2):

   ```bash
   # Override max_steps via Hydra
   bash scripts/job_train.sh -- train.max_steps=50
   ```

   Pull the resulting `trainer_state.json` and run `python scripts/make_plots.py`. The reward curve should now show **any non-zero variance** — even if it's only `0.05 → 0.18`. That alone is a publishable curve.

3. **Decision gate:**
   - If 50-step curve moves: launch the full 400-step run on `a10g-large` (≈$10–15) and replace the plots on `Pratham-math/fathom-1.5b-grpo/plots/*`.
   - If 50-step curve is still flat: stop. Do not spend more credits. Switch to the **honest fallback** in §A.5.

### A.5 Honest fallback (use only if A.4 step 3 still shows flat reward)

If the curve still doesn't move, do not fake it. Re-frame the README to claim what is actually true: **"the SFT phase taught the format; GRPO did not converge in our budget; the env, reward, and pipeline are nonetheless complete and reproducible."** This is genuinely a publishable result — most hackathon submissions don't even get SFT working. The judges' rubric awards points for "showing improvement in rewards" (20%); SFT loss `3.2 → 0.29` and token accuracy `0.46 → 0.93` are improvements. Lead with those plots; relegate the GRPO curve to a section titled "What we learned about reward design."

---

## 3. Problem B — No HF Space UI

### B.1 What the user sees

Hitting <https://Pratham-math-fathom-env.hf.space/> returns `{"detail":"Not Found"}`. There is no landing page. Judges who don't know to append `/healthz` or `/docs` see a blank 404. The Streamlit demo at `viz/app.py` exists locally but has never been deployed and is full of placeholder data anyway (sample tree literal at line 87, fake `[0.10, 0.18, 0.28, ...]` reward sparkline at line 191).

### B.2 Two-Space architecture (do this)

The OpenEnv contract requires the env Space to expose `/reset`, `/step`, etc. as JSON — that's correct, do not change it. But judges need a UI. **Solution: deploy a second Space (Streamlit SDK) that calls the env Space.** This is the canonical pattern in the OpenEnv hackathon submissions (the env Space is the "engine"; the demo Space is the "showroom").

| Space | URL | Purpose | SDK | What changes |
|-------|-----|---------|-----|--------------|
| `Pratham-math/fathom-env` | `Pratham-math-fathom-env.hf.space` | OpenEnv JSON server | Docker | Add a `GET /` HTML index page (B.3) |
| `Pratham-math/fathom-demo` (NEW) | `Pratham-math-fathom-demo.hf.space` | Streamlit UI for judges | Streamlit | New space, scaffolded from `viz/app.py` (B.4) |

### B.3 Patch the env Space — add a root index page

Edit `env/server/app.py` so a judge hitting the bare URL gets a useful HTML response, not a 404. Add this route **before** `@app.get("/healthz")`:

```python
from fastapi.responses import HTMLResponse

INDEX_HTML = """<!DOCTYPE html>
<html><head><meta charset="utf-8"><title>FATHOM Env Server</title>
<style>body{font-family:system-ui,sans-serif;max-width:760px;margin:40px auto;padding:0 20px;line-height:1.55;color:#111}
code{background:#f4f4f5;padding:2px 6px;border-radius:4px}
a{color:#4338ca}.tag{display:inline-block;padding:2px 8px;border-radius:999px;background:#eef2ff;color:#4338ca;font-size:12px;margin-right:6px}</style></head>
<body>
<h1>FATHOM Env Server <span class="tag">OpenEnv 0.2.3</span><span class="tag">Docker</span></h1>
<p><b>FATHOM</b> is the first openly-published OpenEnv RL environment that teaches a small language model to use a recursive-LM scaffold (Python REPL + recursive <code>llm()</code> calls) for long-context QA. Submission for the Meta × PyTorch × Hugging Face OpenEnv Hackathon Grand Finale, Bangalore, Apr 25–26 2026.</p>
<h2>Endpoints</h2>
<ul>
  <li><a href="/healthz"><code>GET /healthz</code></a> — liveness probe</li>
  <li><code>POST /reset</code> — start an episode (try via <a href="/docs">/docs</a>)</li>
  <li><code>POST /step</code> — execute REPL or llm() action</li>
  <li><a href="/state"><code>GET /state</code></a> — sanitized episode state</li>
  <li><a href="/docs"><code>GET /docs</code></a> — interactive OpenAPI</li>
</ul>
<h2>See also</h2>
<ul>
  <li><b>Demo UI:</b> <a href="https://Pratham-math-fathom-demo.hf.space">fathom-demo</a> (Streamlit)</li>
  <li><b>Trained model + plots:</b> <a href="https://huggingface.co/Pratham-math/fathom-1.5b-grpo">Pratham-math/fathom-1.5b-grpo</a></li>
  <li><b>Code repo:</b> <a href="https://huggingface.co/Pratham-math/fathom-code">Pratham-math/fathom-code</a></li>
  <li><b>Colab reproducer:</b> <code>notebooks/fathom_train.ipynb</code> in the code repo</li>
</ul>
</body></html>"""

@app.get("/", response_class=HTMLResponse)
def index() -> HTMLResponse:
    return HTMLResponse(content=INDEX_HTML)
```

Commit with `feat(space): add HTML index for judge first-impression`. Push to the env Space:

```bash
python scripts/deploy_space.py    # already wired to push env/ to fathom-env Space
```

Verify:

```bash
curl -sS https://Pratham-math-fathom-env.hf.space/ | head -20    # should be HTML, not 404
```

### B.4 Build and deploy the Streamlit demo Space

Create the demo Space programmatically:

```bash
mkdir -p space_demo
```

Files to create under `space_demo/`:

**`space_demo/README.md`** (Streamlit Space frontmatter):

```markdown
---
title: FATHOM Demo
emoji: 🧠
colorFrom: indigo
colorTo: purple
sdk: streamlit
sdk_version: 1.39.0
app_file: app.py
pinned: true
license: apache-2.0
---

# FATHOM Demo

Interactive UI for the FATHOM recursive-LM environment. Backed by [Pratham-math/fathom-env](https://huggingface.co/spaces/Pratham-math/fathom-env).
```

**`space_demo/requirements.txt`**:

```
streamlit>=1.39,<2.0
plotly>=5.24,<6.0
httpx>=0.27,<1.0
huggingface_hub>=0.28
pandas>=2.0
```

**`space_demo/app.py`** — port `viz/app.py` here, but **replace placeholder data with real artifacts**. Concretely:

1. **Reward composition pie** — keep, it's accurate.
2. **Recursion tree** (column 1) — replace the literal `sample_tree = {...}` with a live call to `https://Pratham-math-fathom-env.hf.space/reset` then `/step`, capturing the actual REPL trace from one episode. Cache it (`@st.cache_data(ttl=3600)`) so judges don't hammer the env. If the live call fails, fall back to a **clearly-labeled** "example trace" (do not pretend it's live).
3. **Pareto frontier** (column 2) — the current `[0.62, 0.61, 0.58, 0.52, 0.44]` numbers are fabricated. Either:
   - (a) Generate a real one by running the merged_16bit model from `Pratham-math/fathom-1.5b-grpo` against `data/eval.jsonl` at 5 different α values, or
   - (b) Remove this column and replace with an "Eval results" table reading `outputs/eval_*.json` if it exists, or
   - (c) Hide column 2 entirely and widen columns 1 + 3.

   **Pick (b) or (c) if you have <30 min.** Do not ship fabricated numbers.
4. **W&B iframe** (column 3) — set `WANDB_RUN_URL=https://wandb.ai/pratham-alwar05-indian-institute-of-information-technolo/huggingface/runs/sy1tqun0` in the Space's "Variables and secrets" panel so the iframe renders the real run.

Add a top banner cell:

```python
st.markdown(
    f"**Live env:** [Pratham-math/fathom-env]({os.environ.get('FATHOM_SPACE_URL','https://Pratham-math-fathom-env.hf.space')}) "
    f"· **Trained model:** [Pratham-math/fathom-1.5b-grpo](https://huggingface.co/Pratham-math/fathom-1.5b-grpo) "
    f"· **W&B:** [run sy1tqun0](https://wandb.ai/pratham-alwar05-indian-institute-of-information-technolo/huggingface/runs/sy1tqun0)"
)
```

Deploy:

```bash
cd space_demo
huggingface-cli login --token $HF_TOKEN
huggingface-cli repo create fathom-demo --type space --space_sdk streamlit
git init
git remote add origin https://Pratham-math:$HF_TOKEN@huggingface.co/spaces/Pratham-math/fathom-demo
git add -A && git commit -m "feat: initial fathom-demo Space"
git push -u origin main
```

Confirm at <https://Pratham-math-fathom-demo.hf.space> — you should see Streamlit boot in ~2 minutes. **Set the `FATHOM_SPACE_URL` and `WANDB_RUN_URL` Space variables** in the HF UI (Settings → Variables and secrets).

### B.5 Update README submission table

In `README.md` lines 11–22 (the Submission Links table), add a row:

```markdown
| **Demo UI (Streamlit Space)** | <https://huggingface.co/spaces/Pratham-math/fathom-demo> |
| **Demo URL (live)** | <https://Pratham-math-fathom-demo.hf.space> |
```

---

## 4. Problem C — Missing Materials

The hackathon rubric explicitly lists these as **non-negotiable**. From the prompt:

> A short writeup: a mini-blog on Hugging Face or a < 2 minute video on YouTube explaining what your environment does and what you trained, or a short slide deck of presentation. Please make sure that all materials are linked from your README file so that judges can access them easily.

**Status:**

| Item | Required? | Status | Action |
|------|-----------|--------|--------|
| OpenEnv (latest) used | ✅ required | DONE (`openenv-core>=0.2.3`) | none |
| Working Unsloth/TRL training script | ✅ required | DONE (`train/grpo.py`) | none |
| Colab notebook | ✅ "ideally" | DONE-ish (`notebooks/fathom_train.ipynb`) | C.1 verify dataset paths |
| Loss + reward plots from a real run | ✅ required | DONE (10 PNGs on model repo) | A.4 will replace if curve moves |
| Mini-blog OR <2-min video OR slide deck | ✅ **NON-NEGOTIABLE** | **MISSING** | **C.2** |
| HF Space deployed | ✅ required | DONE (`fathom-env`) | B.3 + B.4 enrich |
| README motivation + env + results | ✅ required | DONE | C.3 polish |
| README links to Space + materials | ✅ required | PARTIAL | C.3 |
| **No big video files in env submission** | ✅ required | OK (no videos in repo) | none |

### C.1 Fix the Colab notebook dataset path

`notebooks/fathom_train.ipynb` cell-6 calls `hf_hub_download(repo_id='Pratham-math/fathom-code', filename='data/train.jsonl', ...)`. **Verify those files actually live on `Pratham-math/fathom-code`** — they may not. Run:

```bash
curl -sS "https://huggingface.co/api/models/Pratham-math/fathom-code/tree/main/data" | python -m json.tool
```

If `data/train.jsonl`, `data/eval.jsonl`, and `data/sft_traces.jsonl` are missing, push them:

```bash
huggingface-cli upload Pratham-math/fathom-code data/ data/ --repo-type=model
```

Re-run cell 6 in a Colab to confirm. (You can do this without a GPU — cells 1–5 only need CPU.)

### C.2 Create the mini-blog (fastest of the three options — do this)

A 600-word HF mini-blog beats a video for our time budget. Create it on the Hugging Face Hub:

```bash
huggingface-cli repo create fathom-blog --type space --space_sdk static
```

Then push a single `index.html` (or use the existing `assets/BLOG_DRAFT.md` if it's already drafted — check first with `cat assets/BLOG_DRAFT.md`). Required structure:

1. **Hook** (1 paragraph) — why teach a small model to recurse instead of buying a longer-context one
2. **Environment** (1 paragraph + screenshot of the demo Space) — REPL + `llm()` primitive, deterministic verifier, depth-2 cap
3. **Reward design** (1 paragraph + the reward-composition pie image) — 4 components, anti-hacking audit
4. **Training** (1 paragraph + the SFT loss curve and GRPO reward curve) — be **honest** about the GRPO curve. Frame the v1 flat-line as a finding ("our gate was multiplicative; this is what GRPO collapse looks like"); show the v2 curve underneath (after A.4 retrain) if it moved.
5. **Reproduce** (1 paragraph) — link to Colab notebook + HF Space + model repo
6. **Footer** — names, hackathon, license

Add to `README.md` Submission Links:

```markdown
| **Mini-blog** | <https://huggingface.co/spaces/Pratham-math/fathom-blog> |
```

If the user has already drafted `assets/BLOG_DRAFT.md`, port it into the Space's `index.html` with minimal styling — don't rewrite from scratch.

### C.3 GitHub mirror

```bash
cd C:/Users/prath/OneDrive/Desktop/Hackathons/Meta_finale
gh auth login    # if not already
gh repo create Pratham-math/fathom --public --source=. --remote=github --push
```

Then update README line 17 to:

```markdown
| **Code repo (GitHub mirror)** | <https://github.com/Pratham-math/fathom> |
```

Delete the stale `_to be added — see GITHUB_URL.txt once mirrored_` line.

### C.4 Submission-link block — final state

After C.1–C.3 + B.5, the README's "Submission Links (Judges Start Here)" table must contain **all** of:

- ✅ Environment Space (Hub page) + Endpoint URL + Health check
- ✅ Demo UI Space + Demo URL (NEW)
- ✅ Code repo (HF) + GitHub mirror (NEW)
- ✅ Trained model + plots
- ✅ Colab notebook
- ✅ Mini-blog (NEW)
- ✅ W&B run

Run `python scripts/submission_preflight.py` after the README edits and confirm `Submission package looks judge-ready.`

---

## 5. Problem D — Truth-in-Advertising

### D.1 What's misleading

The README, `CLAUDE.md`, and `assets/architecture.png` all imply that the model **uses** the REPL and recursive `llm()` calls **during GRPO training**. It does not. Read `train/grpo.py:153-169`:

```python
# Why no `env=` / `environment_url=` / `environment_factory` kwargs?
# - TRL 1.2.0's GRPOTrainer.__init__ only accepts env interaction via
#   `tools=` (needs transformers>=5.0), `environment_factory=`
#   (needs transformers>=5.2), or `rollout_func=`. We're on
#   transformers==4.56.2, so the first two raise. The third requires a
#   custom multi-turn rollout implementation we don't have time to
#   harden.
# - Our reward function (rewards.compose) operates on (prompt, completion,
#   gold_answer, prompt_token_count, llm_call_count) — zero env
#   interaction needed.
```

So:

- The **env exists** and is deployed (rubric requirement met).
- The **env is used at inference time** in the demo Space (judges can run a recursive episode).
- The **env is NOT used at training time**. GRPO is single-turn prompt → completion → deterministic reward.

A judge who reads code may flag this as inconsistent with the README's repeated claims about "teaching the model to use the REPL/recursion." That's a goodwill hit we can avoid with one paragraph of plain language.

### D.2 Required README edit

In `README.md`, just after the Architecture section (around line 33), insert this paragraph **verbatim**:

```markdown
### A note on the role of the env in training

TRL 1.2.0 with `transformers==4.56.2` does not yet expose multi-turn env-tool calls inside `GRPOTrainer.train()` (the `tools=` / `environment_factory=` kwargs require `transformers>=5.0`, and a custom `rollout_func=` was outside our time budget). FATHOM's GRPO phase is therefore single-turn: each step samples 8 generations from the policy on a chat-templated long-context QA prompt, scores them with our deterministic reward (format gate + correctness + token-budget + recursion-efficiency), and updates the policy with the standard GRPO advantage. **The env is exercised end-to-end at inference time** — the demo Space runs full multi-turn REPL + recursive `llm()` episodes against the trained model. Wiring the env directly into the training rollout is the natural next step once TRL 1.3 / transformers 5 ships.
```

This is honest, it preempts the obvious code-reading critique, and it actually **reframes our submission as forward-looking** rather than incomplete.

### D.3 Architecture image touch-up (optional, only if time)

`assets/architecture.png` shows arrows from "GRPOTrainer" to "REPL" and "llm()". Either:

- (a) Edit the source `assets/architecture.mmd` (Mermaid) so those arrows are dashed and labeled `inference-time only`, or
- (b) Skip this if you've already done D.2 — the README paragraph carries enough context.

---

## 6. Execution Order (with checkpoints)

| Step | Action | Time | Cost | Checkpoint |
|------|--------|------|------|------------|
| 1 | Read this brief, run `git status`, confirm clean working tree | 5 min | $0 | `git status` clean |
| 2 | Apply A.3.1 (prompt alignment) + A.3.2 (soft format) + A.3.3 (assert) | 30 min | $0 | CPU dry-run prints `PREFIX_MATCH: True` |
| 3 | Commit: `fix(grpo): align prompt with SFT, soft format reward` | 5 min | $0 | git log shows commit |
| 4 | Run smoke on HF Jobs `a10g-large` | 5 min | ~$0.10 | `outputs/smoke/SMOKE_RESULT.md` GO |
| 5 | 50-step GRPO trial run | 20 min | ~$2 | reward_curve.png shows movement |
| 6 | **Decision gate** — full retrain or honest fallback | — | — | see A.4 step 3 |
| 7 | Apply B.3 (env Space index) + push | 15 min | $0 | `curl /` returns HTML |
| 8 | Build B.4 (Streamlit demo Space) + push | 60 min | $0 | demo URL renders |
| 9 | Apply C.2 (mini-blog) | 30 min | $0 | blog Space live |
| 10 | Apply C.3 (GitHub mirror) | 5 min | $0 | GH repo public |
| 11 | Apply D.2 (README clarity paragraph) | 5 min | $0 | README diff |
| 12 | Run `python scripts/submission_preflight.py` | 1 min | $0 | "judge-ready" message |
| 13 | Refresh README submission table per C.4 | 10 min | $0 | all rows filled |
| 14 | Final commit + push to HF master + GH main | 5 min | $0 | both remotes in sync |
| 15 | Hit every link in the README from a fresh browser | 10 min | $0 | nothing 404s |

**Total: ~3.5 h of work + ~$2–15 of cloud GPU depending on retrain decision.**

---

## 7. Things You Might Be Tempted To Do — Don't

- ❌ **Upgrade `trl`/`transformers`/`unsloth` to enable env-tool calls in training.** This is a 2-day refactor with high failure risk. The D.2 paragraph defuses the critique without code changes.
- ❌ **Switch reward composition from weighted-sum to product or RLHF-style ranking.** The composition is fine; the format gate was the bug.
- ❌ **Train a 3B model "for better optics."** The CLAUDE.md is explicit that 1.5B was the deliberate choice. Sticking with 1.5B is part of the story (small-model recursion).
- ❌ **Increase `max_completion_length` past 2048.** STACK §10.3 calls this an anti-pattern. 2048 is fine.
- ❌ **Move from `vllm_mode='colocate'` to `'server'`.** STACK §10.4 + TRL #4543 — this breaks multi-turn. We don't even use multi-turn in training, but `colocate` is also the cheaper option memory-wise.
- ❌ **Refactor `env/server/environment.py`.** It's stable, audited, and shipped. Touch nothing inside `env/` except `app.py` for the index route.
- ❌ **Delete the v1 flat-line GRPO plot.** Honest evidence is part of the storytelling. Either replace with v2 (if A.4 succeeds) or annotate (if A.5 fallback).
- ❌ **Generate fake Pareto numbers because the demo Space looks empty.** Judges who notice are merciless. Either compute real numbers from `data/eval.jsonl` against the merged model or hide the column.
- ❌ **Run `pip install -U` of anything inside the venue venv.** The G10/G12 smoke gates passed on the current pin set; any upgrade voids that.

---

## 8. Reproduction Recipes (use when verifying)

### 8.1 Verify the flat-reward bug is real

```bash
# Convert the UTF-16 log Cursor wrote, then count how many steps had reward != 0
iconv -f UTF-16LE -t UTF-8 job9b_full.log 2>/dev/null \
  | grep -oE "'rewards/_instrumented_reward_fn/mean': [0-9.]+" \
  | sort -u
# Expected: only "'rewards/.../mean': 0.0" — confirms 100% flat
```

### 8.2 Verify the prompt-shape mismatch

```bash
python - <<'PY'
import json
sft = json.loads(open("data/sft_traces.jsonl", encoding="utf-8").readline())
print("SFT user content head:", repr(sft["messages"][1]["content"][:80]))
# Expected: starts with "Question: "

train = json.loads(open("data/train.jsonl", encoding="utf-8").readline())
# Simulate train/grpo.py's _to_prompt user content (PRE-FIX)
print("GRPO user content head (pre-fix):", repr(f"Context:\n{train['context'][:60]}\n\n{train['prompt']}"[:80]))
# Expected: starts with "Context:\n" — confirms the drift
PY
```

### 8.3 After A.3 fixes, dry-run the smoke test locally

```bash
python -m uvicorn env.server.app:app --host 0.0.0.0 --port 8001 &
sleep 5
python -m train.smoke_test --env-url http://localhost:8001
```

Expected: `outputs/smoke/SMOKE_RESULT.md` shows VERDICT: GO.

### 8.4 Verify Space deploy succeeded

```bash
for url in "/" "/healthz" "/state" "/docs"; do
  printf "GET $url → "
  curl -s -o /dev/null -w "%{http_code}\n" "https://Pratham-math-fathom-env.hf.space$url"
done
# Expected: 200, 200, 200, 200 — currently / returns 404.
```

---

## 9. Glossary (Antigravity is cold; here's the cheat sheet)

- **OpenEnv** — Meta's standard for RL environments. Defines the JSON contract `/reset`, `/step`, `/state`, `/healthz`. We pin `openenv-core>=0.2.3,<0.3`.
- **GRPO** — Group Relative Policy Optimization. The trainer samples N completions (here 8), computes per-group advantages (relative to group mean/std), and updates the policy. **Critical:** if all N completions get the same reward, std=0 → advantage=0 → no update. That's exactly our v1 failure mode.
- **TRL** — Hugging Face's RLHF/GRPO trainer library. We're on `trl==1.2.0`.
- **Unsloth** — Memory-efficient LoRA + 4-bit loader. We use `Qwen2.5-Coder-1.5B-Instruct-bnb-4bit` + LoRA r=16.
- **RLM (Recursive Language Model)** — a scaffold where an LM can call itself recursively on document chunks. We cap depth at 2 in training, 4 at demo.
- **Format gate** — the `<answer>...</answer>` regex check. Was multiplicative (binary 0/1), now should be additive (+0.10 bonus).
- **vLLM colocate** — vLLM runs in the same process as TRL trainer, sharing the GPU. Required for multi-turn (we don't use multi-turn in training but colocate is also cheaper memory-wise).

---

## 10. Definition of Done

You are done when **all** of these are true at once:

- [ ] `python scripts/submission_preflight.py` prints `Preflight passed. Submission package looks judge-ready.`
- [ ] `curl -s -o /dev/null -w "%{http_code}" https://Pratham-math-fathom-env.hf.space/` returns **200** (not 404).
- [ ] `https://Pratham-math-fathom-demo.hf.space` renders a Streamlit page in <60 s with no placeholder/fake numbers visible.
- [ ] README's "Submission Links" table has zero `_to be added_` placeholders.
- [ ] Mini-blog space is live and linked from README.
- [ ] GitHub mirror is live and linked from README.
- [ ] `outputs/plots/grpo_reward.png` either (a) shows non-zero variance after A.4 retrain, or (b) is honestly framed in README as v1 with diagnosis (per A.5 fallback).
- [ ] D.2 paragraph appears in README — env's training-vs-inference role is explicit.
- [ ] `git status` is clean on `master`; remote HF master + GitHub main are pushed and in sync.
- [ ] Smoke test green on HF Jobs (`outputs/smoke/SMOKE_RESULT.md` GO timestamp within last 24 h).

If any item is unchecked, you are **not** done. Do not declare victory and stop.

---

## 11. Hand-back Format

When you finish, append a single block to the bottom of this file:

```
## ANTIGRAVITY SESSION RESULT — <YYYY-MM-DD HH:MM IST>

- A (reward curve): <fixed and retrained / honestly reframed / blocked because X>
- B (Space UI): <env index + demo space deployed / partial / blocked>
- C (materials): <blog + GH mirror linked / partial / blocked>
- D (truth-in-advertising): <paragraph in / not in>
- Final preflight: <PASS / FAIL+reason>
- Cloud spend: $<x>
- Commits: <list of commit short-shas>

Open risks for the user before submission:
- ...
```

Stop after that block. Do not push the brief itself; the user committed it.