Spaces:

hamoodey
/

floorplan-graph-rl

Running

App Files Files Community

floorplan-graph-rl / README.md

hamoodey

update

baf8832 verified 1 day ago

preview code

Raw History Blame Contribute Delete

14.6 kB

---
title: Floorplan Graph RL
emoji: 🏠
colorFrom: blue
colorTo: green
sdk: static
pinned: false
license: cc-by-4.0
short_description: Beam learns to read real floor plans by writing code
thumbnail: https://huggingface.co/spaces/hamoodey/floorplan-graph-rl/resolve/main/img/hero.png
---
**Beam can't see an image. So it learned to write its own eyes.**

**A text-only Beam agent writes its own computer-vision pipeline, learns from exact grader feedback, and improves without weight updates.** Its best 10-round run reaches **0.179 held-out partial credit** on 64 unseen real apartment plans and **0.803 room F1 on clean plans**.

**Beam actually learned:** after the grader reported zero rooms on scans, Beam diagnosed its thresholding failure and adopted adaptive thresholding. Public partial credit jumped from **0.022 to 0.179**.

**The problem.** Many floor plans exist only as images or scans. A room graph (rooms + doors) answers "can a wheelchair reach every room?", "how do I get out?" and "how many rooms?". This environment teaches a text-only model to build that graph from a real plan, and grades it exactly.

**In one screen.**
- **A novel Beam loop:** Beam writes a complete vision program, the environment executes it on real plans, and Beam uses exact rewards and failure reports to revise it. Every prompt, reasoning trace, program, and score is published.
- **Measured improvement:** the 10-round run rises from **0.000 to 0.182 public partial credit**; the selected round scores **0.179 on 64 held-out plans from other buildings**. A 46-line general tips playbook also lifts Beam low from **0.032 to 0.075** held-out (3 repeats each; wide spread).
- **A reusable, unsaturated RL environment:** `reset()` / `step(code)`, strict and dense rewards, 24 practice + 64 held-out plans, Docker grading, and a GRPO example. Strict reward remains **0.000** versus **0.242** for the reference: rooms are emerging; doors are the open research problem.

![Input plan, true room graph, Beam's graph, route to exit](https://huggingface.co/spaces/hamoodey/floorplan-graph-rl/resolve/main/img/hero.png)

| Evidence | Result |
|---|---|
| Beam low, 10 feedback rounds | **0.000 → 0.182 public partial**; **0.179 held-out partial** |
| General tips playbook, 3 repeats each | **0.032 → 0.075 held-out partial** |
| Clean-plan room recovery, best 10-round run | **0.803 room F1** |
| Strict held-out reward | **0.000** for every Beam setup; **0.242** reference |

![Beam's rooms per round](https://huggingface.co/spaces/hamoodey/floorplan-graph-rl/resolve/main/img/rounds.gif) ![Held-out partial credit by setup](https://huggingface.co/spaces/hamoodey/floorplan-graph-rl/resolve/main/img/bars.png)

![Score per round](https://huggingface.co/spaces/hamoodey/floorplan-graph-rl/resolve/main/img/rounds-chart.png)

# Floorplan Graph RL: an RL environment that teaches Beam to read real apartment floor plans by writing code

Space (results, charts, plan overlays, wheelchair route): https://huggingface.co/spaces/hamoodey/floorplan-graph-rl
Dataset (this repo): https://huggingface.co/datasets/hamoodey/floorplan-graph-rl


## What this entry is
- **An RL environment for Beam.** `FloorplanEnv.reset()` gives the task prompt; `step(code)` runs Beam's `run.py` and returns the strict reward, a dense partial-credit reward and per-style feedback (room F1, door F1, top failures).
- **Practice and held-out sets.** 24 public practice plans give the feedback; 64 held-out plans from other buildings give the final score and never feed back.
- **Runnable.** Docker files for the sandbox and the grader are included (see How to run).
- **Benchmark evidence.** The Beam and Kimi K3 runs below show what the environment measures.

We did not update Beam's weights; in these runs Beam improves only in-context, by retrying with the environment's feedback. The environment is ready for RL fine-tuning of Beam (open weights).

## What it is
A text-only model cannot see an image. It can still read one, by writing a program that does.
This environment gives the model a raster floor plan of a real apartment and asks for a Python program, `run.py`,
that recovers the **room graph**: every room as a pixel polygon and every door as an edge between two rooms (or a room and `outside`).
Plans come from real Swiss apartments, drawn in four styles: clean; mixed wall drawings with furniture, labels and dimension chains;
clutter on a rotated, blurred, noisy scan; and mixed walls on such a scan.

A recovered room graph is useful on its own: it answers "how do I get from the bedroom to the exit", "is every room reachable
through doors wide enough for a wheelchair", "how many rooms does this flat have". The Space contrasts routes from Beam's recovered graph with the true graph; Beam's door recovery is not yet reliable enough for a trustworthy route.

## How Beam is used
Beam (`Beam-1.0-501B-A23B`, Reflection API, `reasoning_effort` low and high) is the policy:
1. `env.reset()` returns the task prompt: the instruction, the full I/O contract (`task/workspace/README.md`) and the style guide (`task/workspace/docs/STYLE-GUIDE.md`).
2. Beam answers with one complete `run.py`.
3. `env.step(code)` runs that program on the 24 public plans in a sandbox (30 s CPU per plan), scores it, and returns the reward, partial credit and a failure report (crashes, timeouts, invalid output, low IoU rooms, missed doors).
4. Beam gets the feedback and its previous code and writes a better `run.py`. Rounds repeat.
5. The best public round is graded once on the 64 held-out plans (buildings never seen in the public set; published in `task/hidden/` with answers, for reproducibility). The held-out plans never feed back into the loop.

Every prompt, response, reasoning trace, program and score per round is in `runs/`. The same loop also runs Kimi K3 as a comparison, and an earlier Opus 5.5 two-hour agent run (0.032 held-out) is listed as a baseline.

## Reward
- **Per plan:** rooms are matched one to one (Hungarian) at IoU >= 0.7; a door counts when both its rooms match and its centre lies within 0.5 m of the true door. Plan score = room F1 x door F1.
- **Strict reward (the official score):** 16 clean plans form a gate, g = min(1, mean clean score / 0.6); reward = g x mean score over the 48 hard plans (16 per hard style).
- **Partial credit (dense RL signal):** mean over plans of (room F1 + door F1) / 2. The strict reward is near zero for most early programs; partial credit still moves when a program gets rooms right but misses doors, so the model sees progress.

## What generalizes for anyone using Beam on code-writing tasks
- Feedback beats a single try: graded retries took Beam from 0.000 (round 1) to 0.179 held-out partial credit with no weight change.
- Write down what worked: a 46-line tips playbook more than doubled the held-out score (0.032 -> 0.075, 3 repeats each).
- Grade on data the feedback never saw: 64 held-out plans from other buildings keep the gain honest.
- Score partial credit as well as strict: strict was 0.000 in every run, so only partial credit shows learning.

## Beam reading its failure report (reasoning_content, Beam low, 10-round run)
> **Round 5** (practice-plan partial credit 0.022 -> 0.179): "My previous approach using simple thresholding produced 0 rooms on scan styles and poor results. I need a more robust approach."
> **Round 7** (practice-plan partial credit 0.078 -> 0.174): "The previous attempt has low IoU due to morphological closing causing over-merge and room splitting. Let me focus on better wall detection and not over-engineer doors yet."
Full traces: `runs/beam-low-best10/round-5/reasoning.md`, `runs/beam-low-best10/round-7/reasoning.md`.
## Results
| Model | Rounds | Best public strict | Best public partial | Held-out strict (64 plans) | Held-out partial | Best round |
|---|---|---|---|---|---|---|
| Beam low effort, 3 rounds (repeat 1) | 3 | 0.000 | 0.000 | 0.000 | 0.000 | round 1 |
| Beam low effort, 3 rounds (repeat 2) | 3 | 0.000 | 0.026 | 0.000 | 0.035 | round 3 |
| Beam low effort, 3 rounds (repeat 3) | 3 | 0.000 | 0.058 | 0.000 | 0.061 | round 2 |
| Beam low + playbook, 3 rounds (repeat 1) | 3 | 0.000 | 0.101 | 0.000 | 0.102 | round 1 |
| Beam low + playbook, 3 rounds (repeat 2) | 3 | 0.000 | 0.000 | 0.000 | 0.000 | round 1 |
| Beam low + playbook, 3 rounds (repeat 3) | 3 | 0.000 | 0.136 | 0.000 | 0.123 | round 1 |
| Beam high effort, 3 rounds | 3 | 0.000 | 0.010 | 0.000 | 0.016 | round 2 |
| Beam high + playbook, 3 rounds | 3 | 0.000 | 0.000 | 0.000 | 0.000 | round 1 |
| Beam low, 10 rounds, each revises the best code so far | 10 | 0.000 | 0.182 | 0.000 | 0.179 | round 9 |
| Beam high, 5 rounds, best of 4 samples per round | 5 | 0.000 | 0.181 | 0.000 | 0.176 | round 4 |
| Earlier baseline: Opus 5.5, one 2-hour agent run with tools (a different setup; strict score) | - | - | - | 0.032 | - | - |
| Reference solution (hand-written, for scale) | - | - | - | 0.242 | 0.586 | - |
Strict = the official score (below). Partial = the dense RL signal. Public scores drive the feedback; the 64 held-out plans are used only once, for the final score. Per-round scores, plan overlays and the wheelchair route are on the Space.
## Layout
| Path | What |
|---|---|
| `task/instruction.md` | the task statement given to the model |
| `task/workspace/README.md` | full input/output contract |
| `task/workspace/docs/STYLE-GUIDE.md` | drawing styles, door conventions, ranges |
| `task/workspace/public/` | 24 public plans (6 per style): `cases/pNN/plan.png` + `manifest.json`, answers in `answers/pNN.json` |
| `task/workspace/selfcheck/` | self-check, the exact run rules (`runner.py`) and scorer (`score.py`) |
| `task/workspace/submission/run.py` | starting stub (scores 0) |
| `task/tests/` | held-out grader: `verify.py`, `score.py`, the renderer `gen/`, the 64 held-out apartments (`hidden/units/`, geometry), `hidden/index.json`, the render seed and `hidden/expected.json` (hashes of every rendered plan and answer). The verifier renders the 64 held-out plans and their answers from these at grade time. |
| `task/hidden/` | the 64 held-out plans, rendered: `hNN/plan.png`, `manifest.json`, `answer.json` (same format as the public answers), `index.json` (style, scale, counts) |
| `task/environment/` | Docker setup of the sandbox the program runs in (Ubuntu 24.04, Python 3.12, pinned packages) |
| `code/env.py` | `FloorplanEnv`: `reset()` returns the task prompt; `step(code)` returns (reward, partial, feedback, info) |
| `code/grade.py` | grades a `run.py` with the Docker images (public self-check or held-out verifier); prints strict reward, partial credit, per-style F1, failures |
| `code/llm.py`, `code/loop.py` | the model loop (Beam via the Reflection API, Kimi K3) |
| `code/build_space.py` | builds the Space page: table, chart, overlays, networkx wheelchair route |
| `runs/<model>/round-<k>/` | every round: `prompt.md`, `response.md`, `reasoning.md` (Beam's thinking), `run.py`, `public.json` (scores); `runs/<model>/hidden.json` final held-out score |

## Use it for RL
```python
from code.env import FloorplanEnv, extract_code   # run from the repo root; grading needs the two Docker images (How to run)
env = FloorplanEnv(set="public")                  # 24 practice plans; set="hidden" only for the final score
prompt = env.reset()                              # instruction + contract + style guide + answer format
reward, partial, feedback, info = env.step(extract_code(model_reply))   # strict reward, dense reward, per-style report
next_prompt = prompt + "\n\nYour previous run.py:\n" + code + "\n\nGrader feedback:\n" + feedback
```
A GRPO-style reward function (for example TRL's `GRPOTrainer(reward_funcs=[floorplan_reward])`), using the dense partial credit so that early programs still get a signal:
```python
def floorplan_reward(completions, **kwargs):
    out = []
    for c in completions:
        text = c if isinstance(c, str) else c[-1]["content"]
        reward, partial, feedback, info = FloorplanEnv(set="public").step(extract_code(text))
        out.append(partial + reward)   # strict reward adds on top once door F1 > 0
    return out
```
The task is far from saturated: the strict reward is still 0.000 for every model loop here, while a hand-written reference reaches 0.242. There is a lot of room to train.

## Why trust these numbers
The grader is exact (rooms at IoU 0.7, doors within 0.5 m). Feedback uses only the 24 practice plans; the 64 held-out plans are from other buildings and never feed back. Every prompt, reasoning trace, program and score is in `runs/`.

**Known limits.** Kimi K3 did not finish (no reply to its round-1 request in 31 min); Beam high effort finished (held-out 0.016 in a single 3-round run; 0.176 with best of 4 samples per round). Most numbers are one run; the playbook result has 3 repeats with a wide spread. Doors are nearly unsolved (door F1 0.000 on every setup but one). We did not update Beam's weights.

## How to run
```bash
# 1. Build the sandbox and the held-out grader (Docker, linux/amd64)
docker build -t fp-agent task/environment
docker build -t fp-verifier --build-arg TASK_IMAGE=fp-agent task/tests
# 2. Public self-check of your program (24 plans, official runner and scorer)
docker run --rm -v "$PWD/my_submission:/mnt/sub:ro" --entrypoint bash fp-verifier -c \
  'rm -rf /app/submission/* && cp -a /mnt/sub/. /app/submission/ && python3 /app/selfcheck/selfcheck.py'
# 3. Held-out score (renders the 64 held-out plans, runs your program as an unprivileged user, writes /logs/verifier/reward.json)
docker run --rm -v "$PWD/my_submission:/mnt/sub:ro" --entrypoint bash fp-verifier -c \
  'rm -rf /app/submission/* && cp -a /mnt/sub/. /app/submission/ && /tests/test.sh; cat /logs/verifier/reward.json'
# 4. The RL loop (Beam writes run.py, gets scores and failures, retries); needs REFLECTION_API_KEY
#    code/grade.py runs the docker commands above over ssh on FP_HOST (a Docker host with the two images)
python3 code/loop.py --model beam --effort high
```
Without Docker: `cp my_run.py task/workspace/submission/run.py && python3 task/workspace/selfcheck/selfcheck.py` (needs Python 3.12 with numpy, scipy, opencv, scikit-image, shapely, networkx, Pillow).

## Credit and license
Plan geometry: **Swiss Dwellings v3.0.0** (Standfest et al.), Zenodo, DOI [10.5281/zenodo.7788422](https://doi.org/10.5281/zenodo.7788422), CC BY 4.0.
The plans here are re-rendered from that geometry. This repo is released under CC BY 4.0.
Built by Hamood (hamoodey) for Reflection's "Build with Beam" contest.

	---
	title: Floorplan Graph RL
	emoji: 🏠
	colorFrom: blue
	colorTo: green
	sdk: static
	pinned: false
	license: cc-by-4.0
	short_description: Beam learns to read real floor plans by writing code
	thumbnail: https://huggingface.co/spaces/hamoodey/floorplan-graph-rl/resolve/main/img/hero.png
	---
	Beam can't see an image. So it learned to write its own eyes.

	A text-only Beam agent writes its own computer-vision pipeline, learns from exact grader feedback, and improves without weight updates. Its best 10-round run reaches 0.179 held-out partial credit on 64 unseen real apartment plans and 0.803 room F1 on clean plans.

	Beam actually learned: after the grader reported zero rooms on scans, Beam diagnosed its thresholding failure and adopted adaptive thresholding. Public partial credit jumped from 0.022 to 0.179.

	The problem. Many floor plans exist only as images or scans. A room graph (rooms + doors) answers "can a wheelchair reach every room?", "how do I get out?" and "how many rooms?". This environment teaches a text-only model to build that graph from a real plan, and grades it exactly.

	In one screen.
	- A novel Beam loop: Beam writes a complete vision program, the environment executes it on real plans, and Beam uses exact rewards and failure reports to revise it. Every prompt, reasoning trace, program, and score is published.
	- Measured improvement: the 10-round run rises from 0.000 to 0.182 public partial credit; the selected round scores 0.179 on 64 held-out plans from other buildings. A 46-line general tips playbook also lifts Beam low from 0.032 to 0.075 held-out (3 repeats each; wide spread).
	- A reusable, unsaturated RL environment: `reset()` / `step(code)`, strict and dense rewards, 24 practice + 64 held-out plans, Docker grading, and a GRPO example. Strict reward remains 0.000 versus 0.242 for the reference: rooms are emerging; doors are the open research problem.

	![Input plan, true room graph, Beam's graph, route to exit](https://huggingface.co/spaces/hamoodey/floorplan-graph-rl/resolve/main/img/hero.png)

	\| Evidence \| Result \|
	\|---\|---\|
	\| Beam low, 10 feedback rounds \| 0.000 → 0.182 public partial; 0.179 held-out partial \|
	\| General tips playbook, 3 repeats each \| 0.032 → 0.075 held-out partial \|
	\| Clean-plan room recovery, best 10-round run \| 0.803 room F1 \|
	\| Strict held-out reward \| 0.000 for every Beam setup; 0.242 reference \|

	![Beam's rooms per round](https://huggingface.co/spaces/hamoodey/floorplan-graph-rl/resolve/main/img/rounds.gif) ![Held-out partial credit by setup](https://huggingface.co/spaces/hamoodey/floorplan-graph-rl/resolve/main/img/bars.png)

	![Score per round](https://huggingface.co/spaces/hamoodey/floorplan-graph-rl/resolve/main/img/rounds-chart.png)

	# Floorplan Graph RL: an RL environment that teaches Beam to read real apartment floor plans by writing code

	Space (results, charts, plan overlays, wheelchair route): https://huggingface.co/spaces/hamoodey/floorplan-graph-rl
	Dataset (this repo): https://huggingface.co/datasets/hamoodey/floorplan-graph-rl


	## What this entry is
	- An RL environment for Beam. `FloorplanEnv.reset()` gives the task prompt; `step(code)` runs Beam's `run.py` and returns the strict reward, a dense partial-credit reward and per-style feedback (room F1, door F1, top failures).
	- Practice and held-out sets. 24 public practice plans give the feedback; 64 held-out plans from other buildings give the final score and never feed back.
	- Runnable. Docker files for the sandbox and the grader are included (see How to run).
	- Benchmark evidence. The Beam and Kimi K3 runs below show what the environment measures.

	We did not update Beam's weights; in these runs Beam improves only in-context, by retrying with the environment's feedback. The environment is ready for RL fine-tuning of Beam (open weights).

	## What it is
	A text-only model cannot see an image. It can still read one, by writing a program that does.
	This environment gives the model a raster floor plan of a real apartment and asks for a Python program, `run.py`,
	that recovers the room graph: every room as a pixel polygon and every door as an edge between two rooms (or a room and `outside`).
	Plans come from real Swiss apartments, drawn in four styles: clean; mixed wall drawings with furniture, labels and dimension chains;
	clutter on a rotated, blurred, noisy scan; and mixed walls on such a scan.

	A recovered room graph is useful on its own: it answers "how do I get from the bedroom to the exit", "is every room reachable
	through doors wide enough for a wheelchair", "how many rooms does this flat have". The Space contrasts routes from Beam's recovered graph with the true graph; Beam's door recovery is not yet reliable enough for a trustworthy route.

	## How Beam is used
	Beam (`Beam-1.0-501B-A23B`, Reflection API, `reasoning_effort` low and high) is the policy:
	1. `env.reset()` returns the task prompt: the instruction, the full I/O contract (`task/workspace/README.md`) and the style guide (`task/workspace/docs/STYLE-GUIDE.md`).
	2. Beam answers with one complete `run.py`.
	3. `env.step(code)` runs that program on the 24 public plans in a sandbox (30 s CPU per plan), scores it, and returns the reward, partial credit and a failure report (crashes, timeouts, invalid output, low IoU rooms, missed doors).
	4. Beam gets the feedback and its previous code and writes a better `run.py`. Rounds repeat.
	5. The best public round is graded once on the 64 held-out plans (buildings never seen in the public set; published in `task/hidden/` with answers, for reproducibility). The held-out plans never feed back into the loop.

	Every prompt, response, reasoning trace, program and score per round is in `runs/`. The same loop also runs Kimi K3 as a comparison, and an earlier Opus 5.5 two-hour agent run (0.032 held-out) is listed as a baseline.

	## Reward
	- Per plan: rooms are matched one to one (Hungarian) at IoU >= 0.7; a door counts when both its rooms match and its centre lies within 0.5 m of the true door. Plan score = room F1 x door F1.
	- Strict reward (the official score): 16 clean plans form a gate, g = min(1, mean clean score / 0.6); reward = g x mean score over the 48 hard plans (16 per hard style).
	- Partial credit (dense RL signal): mean over plans of (room F1 + door F1) / 2. The strict reward is near zero for most early programs; partial credit still moves when a program gets rooms right but misses doors, so the model sees progress.

	## What generalizes for anyone using Beam on code-writing tasks
	- Feedback beats a single try: graded retries took Beam from 0.000 (round 1) to 0.179 held-out partial credit with no weight change.
	- Write down what worked: a 46-line tips playbook more than doubled the held-out score (0.032 -> 0.075, 3 repeats each).
	- Grade on data the feedback never saw: 64 held-out plans from other buildings keep the gain honest.
	- Score partial credit as well as strict: strict was 0.000 in every run, so only partial credit shows learning.

	## Beam reading its failure report (reasoning_content, Beam low, 10-round run)
	> Round 5 (practice-plan partial credit 0.022 -> 0.179): "My previous approach using simple thresholding produced 0 rooms on scan styles and poor results. I need a more robust approach."

	> Round 7 (practice-plan partial credit 0.078 -> 0.174): "The previous attempt has low IoU due to morphological closing causing over-merge and room splitting. Let me focus on better wall detection and not over-engineer doors yet."

	Full traces: `runs/beam-low-best10/round-5/reasoning.md`, `runs/beam-low-best10/round-7/reasoning.md`.

	## Results
	\| Model \| Rounds \| Best public strict \| Best public partial \| Held-out strict (64 plans) \| Held-out partial \| Best round \|
	\|---\|---\|---\|---\|---\|---\|---\|
	\| Beam low effort, 3 rounds (repeat 1) \| 3 \| 0.000 \| 0.000 \| 0.000 \| 0.000 \| round 1 \|
	\| Beam low effort, 3 rounds (repeat 2) \| 3 \| 0.000 \| 0.026 \| 0.000 \| 0.035 \| round 3 \|
	\| Beam low effort, 3 rounds (repeat 3) \| 3 \| 0.000 \| 0.058 \| 0.000 \| 0.061 \| round 2 \|
	\| Beam low + playbook, 3 rounds (repeat 1) \| 3 \| 0.000 \| 0.101 \| 0.000 \| 0.102 \| round 1 \|
	\| Beam low + playbook, 3 rounds (repeat 2) \| 3 \| 0.000 \| 0.000 \| 0.000 \| 0.000 \| round 1 \|
	\| Beam low + playbook, 3 rounds (repeat 3) \| 3 \| 0.000 \| 0.136 \| 0.000 \| 0.123 \| round 1 \|
	\| Beam high effort, 3 rounds \| 3 \| 0.000 \| 0.010 \| 0.000 \| 0.016 \| round 2 \|
	\| Beam high + playbook, 3 rounds \| 3 \| 0.000 \| 0.000 \| 0.000 \| 0.000 \| round 1 \|
	\| Beam low, 10 rounds, each revises the best code so far \| 10 \| 0.000 \| 0.182 \| 0.000 \| 0.179 \| round 9 \|
	\| Beam high, 5 rounds, best of 4 samples per round \| 5 \| 0.000 \| 0.181 \| 0.000 \| 0.176 \| round 4 \|
	\| Earlier baseline: Opus 5.5, one 2-hour agent run with tools (a different setup; strict score) \| - \| - \| - \| 0.032 \| - \| - \|
	\| Reference solution (hand-written, for scale) \| - \| - \| - \| 0.242 \| 0.586 \| - \|

	Strict = the official score (below). Partial = the dense RL signal. Public scores drive the feedback; the 64 held-out plans are used only once, for the final score. Per-round scores, plan overlays and the wheelchair route are on the Space.

	## Layout
	\| Path \| What \|
	\|---\|---\|
	\| `task/instruction.md` \| the task statement given to the model \|
	\| `task/workspace/README.md` \| full input/output contract \|
	\| `task/workspace/docs/STYLE-GUIDE.md` \| drawing styles, door conventions, ranges \|
	\| `task/workspace/public/` \| 24 public plans (6 per style): `cases/pNN/plan.png` + `manifest.json`, answers in `answers/pNN.json` \|
	\| `task/workspace/selfcheck/` \| self-check, the exact run rules (`runner.py`) and scorer (`score.py`) \|
	\| `task/workspace/submission/run.py` \| starting stub (scores 0) \|
	\| `task/tests/` \| held-out grader: `verify.py`, `score.py`, the renderer `gen/`, the 64 held-out apartments (`hidden/units/`, geometry), `hidden/index.json`, the render seed and `hidden/expected.json` (hashes of every rendered plan and answer). The verifier renders the 64 held-out plans and their answers from these at grade time. \|
	\| `task/hidden/` \| the 64 held-out plans, rendered: `hNN/plan.png`, `manifest.json`, `answer.json` (same format as the public answers), `index.json` (style, scale, counts) \|
	\| `task/environment/` \| Docker setup of the sandbox the program runs in (Ubuntu 24.04, Python 3.12, pinned packages) \|
	\| `code/env.py` \| `FloorplanEnv`: `reset()` returns the task prompt; `step(code)` returns (reward, partial, feedback, info) \|
	\| `code/grade.py` \| grades a `run.py` with the Docker images (public self-check or held-out verifier); prints strict reward, partial credit, per-style F1, failures \|
	\| `code/llm.py`, `code/loop.py` \| the model loop (Beam via the Reflection API, Kimi K3) \|
	\| `code/build_space.py` \| builds the Space page: table, chart, overlays, networkx wheelchair route \|
	\| `runs/<model>/round-<k>/` \| every round: `prompt.md`, `response.md`, `reasoning.md` (Beam's thinking), `run.py`, `public.json` (scores); `runs/<model>/hidden.json` final held-out score \|

	## Use it for RL
	```python
	from code.env import FloorplanEnv, extract_code # run from the repo root; grading needs the two Docker images (How to run)
	env = FloorplanEnv(set="public") # 24 practice plans; set="hidden" only for the final score
	prompt = env.reset() # instruction + contract + style guide + answer format
	reward, partial, feedback, info = env.step(extract_code(model_reply)) # strict reward, dense reward, per-style report
	next_prompt = prompt + "\n\nYour previous run.py:\n" + code + "\n\nGrader feedback:\n" + feedback
	```
	A GRPO-style reward function (for example TRL's `GRPOTrainer(reward_funcs=[floorplan_reward])`), using the dense partial credit so that early programs still get a signal:
	```python
	def floorplan_reward(completions, **kwargs):
	out = []
	for c in completions:
	text = c if isinstance(c, str) else c[-1]["content"]
	reward, partial, feedback, info = FloorplanEnv(set="public").step(extract_code(text))
	out.append(partial + reward) # strict reward adds on top once door F1 > 0
	return out
	```
	The task is far from saturated: the strict reward is still 0.000 for every model loop here, while a hand-written reference reaches 0.242. There is a lot of room to train.

	## Why trust these numbers
	The grader is exact (rooms at IoU 0.7, doors within 0.5 m). Feedback uses only the 24 practice plans; the 64 held-out plans are from other buildings and never feed back. Every prompt, reasoning trace, program and score is in `runs/`.

	Known limits. Kimi K3 did not finish (no reply to its round-1 request in 31 min); Beam high effort finished (held-out 0.016 in a single 3-round run; 0.176 with best of 4 samples per round). Most numbers are one run; the playbook result has 3 repeats with a wide spread. Doors are nearly unsolved (door F1 0.000 on every setup but one). We did not update Beam's weights.

	## How to run
	```bash
	# 1. Build the sandbox and the held-out grader (Docker, linux/amd64)
	docker build -t fp-agent task/environment
	docker build -t fp-verifier --build-arg TASK_IMAGE=fp-agent task/tests

	# 2. Public self-check of your program (24 plans, official runner and scorer)
	docker run --rm -v "$PWD/my_submission:/mnt/sub:ro" --entrypoint bash fp-verifier -c \
	'rm -rf /app/submission/* && cp -a /mnt/sub/. /app/submission/ && python3 /app/selfcheck/selfcheck.py'

	# 3. Held-out score (renders the 64 held-out plans, runs your program as an unprivileged user, writes /logs/verifier/reward.json)
	docker run --rm -v "$PWD/my_submission:/mnt/sub:ro" --entrypoint bash fp-verifier -c \
	'rm -rf /app/submission/* && cp -a /mnt/sub/. /app/submission/ && /tests/test.sh; cat /logs/verifier/reward.json'

	# 4. The RL loop (Beam writes run.py, gets scores and failures, retries); needs REFLECTION_API_KEY
	# code/grade.py runs the docker commands above over ssh on FP_HOST (a Docker host with the two images)
	python3 code/loop.py --model beam --effort high
	```
	Without Docker: `cp my_run.py task/workspace/submission/run.py && python3 task/workspace/selfcheck/selfcheck.py` (needs Python 3.12 with numpy, scipy, opencv, scikit-image, shapely, networkx, Pillow).

	## Credit and license
	Plan geometry: Swiss Dwellings v3.0.0 (Standfest et al.), Zenodo, DOI [10.5281/zenodo.7788422](https://doi.org/10.5281/zenodo.7788422), CC BY 4.0.
	The plans here are re-rendered from that geometry. This repo is released under CC BY 4.0.
	Built by Hamood (hamoodey) for Reflection's "Build with Beam" contest.