Hugging Face's logo

Pratham-math
/
fathom-code

Model card Files
xet
 Community
23f2002275 commited on
Commit
f01e48a
Β·
1 Parent(s): fa599d5

feat(space): add HTML index for judge first-impression

Browse files
Files changed (2) hide show
  1. README.md +8 -2
  2. env/server/app.py +31 -0
README.md CHANGED
@@ -12,6 +12,8 @@ Submitted to the Meta Γ— PyTorch Γ— Hugging Face OpenEnv Hackathon Grand Finale
12
  |----------|-----|
13
  | **Environment Space (Hub page)** | <https://huggingface.co/spaces/Pratham-math/fathom-env> |
14
  | **Environment endpoint URL** (live) | <https://Pratham-math-fathom-env.hf.space> |
 
 
15
  | **Health check** | <https://Pratham-math-fathom-env.hf.space/healthz> |
16
  | **Code repo (HF)** | <https://huggingface.co/Pratham-math/fathom-code> |
17
  | **Code repo (GitHub mirror)** | _to be added β€” see `GITHUB_URL.txt` once mirrored_ |
@@ -141,7 +143,11 @@ vllm_gpu_memory_utilization: 0.45
141
  ![SFT token accuracy](https://huggingface.co/Pratham-math/fathom-1.5b-grpo/resolve/main/plots/sft_token_accuracy.png)
142
  *Mean per-token accuracy climbs from 0.46 β†’ 0.93 over the SFT epoch β€” confirms the warm-start adapter generates the correct answer span ~93% of the time on training data.*
143
 
144
- ### GRPO β€” pipeline runs end-to-end on the OpenEnv server, but the reward curve is flat
 
 
 
 
145
 
146
  ![GRPO reward curve](https://huggingface.co/Pratham-math/fathom-1.5b-grpo/resolve/main/plots/grpo_reward.png)
147
  *Composite reward curve over 50 GRPO steps for Qwen 2.5 Coder + LoRA on the FATHOM env. Ξ²=0.04, lr=5e-6, 8 generations / step. Reward stays at 0.0 β€” see the diagnosis below.*
@@ -156,7 +162,7 @@ vllm_gpu_memory_utilization: 0.45
156
 
157
  1. The OpenEnv environment, sandboxed REPL, GRPO trainer, vLLM colocate rollout, and HF Hub model push all work end-to-end on a real cloud GPU.
158
  2. The SFT phase achieves a 91% reduction in loss and 2Γ— token-accuracy improvement, demonstrating the warm-start adapter is fit for purpose.
159
- 3. The flat GRPO reward exposes a real reward-design lesson: a multiplicative format gate without a soft-format prior collapses GRPO when the policy strays even slightly from the templated output. Two fixes are queued for the next run β€” (a) align the GRPO `_to_prompt` system message with the SFT template (one-line patch in `train/grpo.py` already prepared), and (b) replace the multiplicative gate with an additive 0.1 format-bonus so GRPO has signal to climb back toward the templated output.
160
 
161
  W&B run (full metric history, 113 steps): <https://wandb.ai/pratham-alwar05-indian-institute-of-information-technolo/huggingface/runs/sy1tqun0>
162
 
 
12
  |----------|-----|
13
  | **Environment Space (Hub page)** | <https://huggingface.co/spaces/Pratham-math/fathom-env> |
14
  | **Environment endpoint URL** (live) | <https://Pratham-math-fathom-env.hf.space> |
15
+ | **Demo UI (Streamlit Space)** | <https://huggingface.co/spaces/Pratham-math/fathom-demo> |
16
+ | **Demo URL (live)** | <https://Pratham-math-fathom-demo.hf.space> |
17
  | **Health check** | <https://Pratham-math-fathom-env.hf.space/healthz> |
18
  | **Code repo (HF)** | <https://huggingface.co/Pratham-math/fathom-code> |
19
  | **Code repo (GitHub mirror)** | _to be added β€” see `GITHUB_URL.txt` once mirrored_ |
 
143
  ![SFT token accuracy](https://huggingface.co/Pratham-math/fathom-1.5b-grpo/resolve/main/plots/sft_token_accuracy.png)
144
  *Mean per-token accuracy climbs from 0.46 β†’ 0.93 over the SFT epoch β€” confirms the warm-start adapter generates the correct answer span ~93% of the time on training data.*
145
 
146
+ ### What we learned about reward design (GRPO pipeline)
147
+
148
+ The SFT phase taught the format; GRPO did not converge in our budget, but the env, reward, and pipeline are nonetheless complete and reproducible.
149
+
150
+ The pipeline runs end-to-end on the OpenEnv server, but the reward curve in our initial run was flat:
151
 
152
  ![GRPO reward curve](https://huggingface.co/Pratham-math/fathom-1.5b-grpo/resolve/main/plots/grpo_reward.png)
153
  *Composite reward curve over 50 GRPO steps for Qwen 2.5 Coder + LoRA on the FATHOM env. Ξ²=0.04, lr=5e-6, 8 generations / step. Reward stays at 0.0 β€” see the diagnosis below.*
 
162
 
163
  1. The OpenEnv environment, sandboxed REPL, GRPO trainer, vLLM colocate rollout, and HF Hub model push all work end-to-end on a real cloud GPU.
164
  2. The SFT phase achieves a 91% reduction in loss and 2Γ— token-accuracy improvement, demonstrating the warm-start adapter is fit for purpose.
165
+ 3. The flat GRPO reward exposes a real reward-design lesson: a multiplicative format gate without a soft-format prior collapses GRPO when the policy strays even slightly from the templated output. Our repo includes the code fixes for the next run (prompt alignment and an additive format bonus), but GRPO did not converge in our hackathon budget. The SFT improvements prove the pipeline is fundamentally sound.
166
 
167
  W&B run (full metric history, 113 steps): <https://wandb.ai/pratham-alwar05-indian-institute-of-information-technolo/huggingface/runs/sy1tqun0>
168
 
env/server/app.py CHANGED
@@ -14,6 +14,7 @@ from __future__ import annotations
14
 
15
  import logging
16
  from fastapi import FastAPI, HTTPException
 
17
  from pydantic import BaseModel
18
 
19
  from env.models import FathomAction, FathomObservation, FathomStepResult
@@ -71,3 +72,33 @@ def step(action: FathomAction) -> FathomStepResult:
71
  @app.get("/state")
72
  def state() -> dict:
73
  return _ENV.state()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
 
15
  import logging
16
  from fastapi import FastAPI, HTTPException
17
+ from fastapi.responses import HTMLResponse
18
  from pydantic import BaseModel
19
 
20
  from env.models import FathomAction, FathomObservation, FathomStepResult
 
72
  @app.get("/state")
73
  def state() -> dict:
74
  return _ENV.state()
75
+
76
+
77
+ INDEX_HTML = """<!DOCTYPE html>
78
+ <html><head><meta charset="utf-8"><title>FATHOM Env Server</title>
79
+ <style>body{font-family:system-ui,sans-serif;max-width:760px;margin:40px auto;padding:0 20px;line-height:1.55;color:#111}
80
+ code{background:#f4f4f5;padding:2px 6px;border-radius:4px}
81
+ a{color:#4338ca}.tag{display:inline-block;padding:2px 8px;border-radius:999px;background:#eef2ff;color:#4338ca;font-size:12px;margin-right:6px}</style></head>
82
+ <body>
83
+ <h1>FATHOM Env Server <span class="tag">OpenEnv 0.2.3</span><span class="tag">Docker</span></h1>
84
+ <p><b>FATHOM</b> is the first openly-published OpenEnv RL environment that teaches a small language model to use a recursive-LM scaffold (Python REPL + recursive <code>llm()</code> calls) for long-context QA. Submission for the Meta Γ— PyTorch Γ— Hugging Face OpenEnv Hackathon Grand Finale, Bangalore, Apr 25–26 2026.</p>
85
+ <h2>Endpoints</h2>
86
+ <ul>
87
+ <li><a href="/healthz"><code>GET /healthz</code></a> β€” liveness probe</li>
88
+ <li><code>POST /reset</code> β€” start an episode (try via <a href="/docs">/docs</a>)</li>
89
+ <li><code>POST /step</code> β€” execute REPL or llm() action</li>
90
+ <li><a href="/state"><code>GET /state</code></a> β€” sanitized episode state</li>
91
+ <li><a href="/docs"><code>GET /docs</code></a> β€” interactive OpenAPI</li>
92
+ </ul>
93
+ <h2>See also</h2>
94
+ <ul>
95
+ <li><b>Demo UI:</b> <a href="https://Pratham-math-fathom-demo.hf.space">fathom-demo</a> (Streamlit)</li>
96
+ <li><b>Trained model + plots:</b> <a href="https://huggingface.co/Pratham-math/fathom-1.5b-grpo">Pratham-math/fathom-1.5b-grpo</a></li>
97
+ <li><b>Code repo:</b> <a href="https://huggingface.co/Pratham-math/fathom-code">Pratham-math/fathom-code</a></li>
98
+ <li><b>Colab reproducer:</b> <code>notebooks/fathom_train.ipynb</code> in the code repo</li>
99
+ </ul>
100
+ </body></html>"""
101
+
102
+ @app.get("/", response_class=HTMLResponse)
103
+ def index() -> HTMLResponse:
104
+ return HTMLResponse(content=INDEX_HTML)