Blame for README.md

---
title: Floorplan Graph RL
emoji: 🏠
colorFrom: blue
colorTo: green
sdk: static
pinned: false
license: cc-by-4.0
short_description: Beam learns to read real floor plans by writing code
thumbnail: https://huggingface.co/spaces/hamoodey/floorplan-graph-rl/resolve/main/img/hero.png
---
**Beam can't see an image. So it learned to write its own eyes.**

**A text-only Beam agent writes its own computer-vision pipeline, learns from exact grader feedback, and improves without weight updates.** Its best 10-round run reaches **0.179 held-out partial credit** on 64 unseen real apartment plans and **0.803 room F1 on clean plans**.

**Beam actually learned:** after the grader reported zero rooms on scans, Beam diagnosed its thresholding failure and adopted adaptive thresholding. Public partial credit jumped from **0.022 to 0.179**.

**The problem.** Many floor plans exist only as images or scans. A room graph (rooms + doors) answers "can a wheelchair reach every room?", "how do I get out?" and "how many rooms?". This environment teaches a text-only model to build that graph from a real plan, and grades it exactly.

**In one screen.**
- **A novel Beam loop:** Beam writes a complete vision program, the environment executes it on real plans, and Beam uses exact rewards and failure reports to revise it. Every prompt, reasoning trace, program, and score is published.
- **Measured improvement:** the 10-round run rises from **0.000 to 0.182 public partial credit**; the selected round scores **0.179 on 64 held-out plans from other buildings**. A 46-line general tips playbook also lifts Beam low from **0.032 to 0.075** held-out (3 repeats each; wide spread).
- **A reusable, unsaturated RL environment:** `reset()` / `step(code)`, strict and dense rewards, 24 practice + 64 held-out plans, Docker grading, and a GRPO example. Strict reward remains **0.000** versus **0.242** for the reference: rooms are emerging; doors are the open research problem.

![Input plan, true room graph, Beam's graph, route to exit](https://huggingface.co/spaces/hamoodey/floorplan-graph-rl/resolve/main/img/hero.png)

| Evidence | Result |
|---|---|
| Beam low, 10 feedback rounds | **0.000 → 0.182 public partial**; **0.179 held-out partial** |
| General tips playbook, 3 repeats each | **0.032 → 0.075 held-out partial** |
| Clean-plan room recovery, best 10-round run | **0.803 room F1** |
| Strict held-out reward | **0.000** for every Beam setup; **0.242** reference |

![Beam's rooms per round](https://huggingface.co/spaces/hamoodey/floorplan-graph-rl/resolve/main/img/rounds.gif) ![Held-out partial credit by setup](https://huggingface.co/spaces/hamoodey/floorplan-graph-rl/resolve/main/img/bars.png)

![Score per round](https://huggingface.co/spaces/hamoodey/floorplan-graph-rl/resolve/main/img/rounds-chart.png)

# Floorplan Graph RL: an RL environment that teaches Beam to read real apartment floor plans by writing code

Space (results, charts, plan overlays, wheelchair route): https://huggingface.co/spaces/hamoodey/floorplan-graph-rl
Dataset (this repo): https://huggingface.co/datasets/hamoodey/floorplan-graph-rl


## What this entry is
- **An RL environment for Beam.** `FloorplanEnv.reset()` gives the task prompt; `step(code)` runs Beam's `run.py` and returns the strict reward, a dense partial-credit reward and per-style feedback (room F1, door F1, top failures).
- **Practice and held-out sets.** 24 public practice plans give the feedback; 64 held-out plans from other buildings give the final score and never feed back.
- **Runnable.** Docker files for the sandbox and the grader are included (see How to run).
- **Benchmark evidence.** The Beam and Kimi K3 runs below show what the environment measures.

We did not update Beam's weights; in these runs Beam improves only in-context, by retrying with the environment's feedback. The environment is ready for RL fine-tuning of Beam (open weights).

## What it is
A text-only model cannot see an image. It can still read one, by writing a program that does.
This environment gives the model a raster floor plan of a real apartment and asks for a Python program, `run.py`,
that recovers the **room graph**: every room as a pixel polygon and every door as an edge between two rooms (or a room and `outside`).
Plans come from real Swiss apartments, drawn in four styles: clean; mixed wall drawings with furniture, labels and dimension chains;
clutter on a rotated, blurred, noisy scan; and mixed walls on such a scan.

A recovered room graph is useful on its own: it answers "how do I get from the bedroom to the exit", "is every room reachable
through doors wide enough for a wheelchair", "how many rooms does this flat have". The Space contrasts routes from Beam's recovered graph with the true graph; Beam's door recovery is not yet reliable enough for a trustworthy route.

## How Beam is used
Beam (`Beam-1.0-501B-A23B`, Reflection API, `reasoning_effort` low and high) is the policy:
1. `env.reset()` returns the task prompt: the instruction, the full I/O contract (`task/workspace/README.md`) and the style guide (`task/workspace/docs/STYLE-GUIDE.md`).
2. Beam answers with one complete `run.py`.
3. `env.step(code)` runs that program on the 24 public plans in a sandbox (30 s CPU per plan), scores it, and returns the reward, partial credit and a failure report (crashes, timeouts, invalid output, low IoU rooms, missed doors).
4. Beam gets the feedback and its previous code and writes a better `run.py`. Rounds repeat.
5. The best public round is graded once on the 64 held-out plans (buildings never seen in the public set; published in `task/hidden/` with answers, for reproducibility). The held-out plans never feed back into the loop.

Every prompt, response, reasoning trace, program and score per round is in `runs/`. The same loop also runs Kimi K3 as a comparison, and an earlier Opus 5.5 two-hour agent run (0.032 held-out) is listed as a baseline.

## Reward
- **Per plan:** rooms are matched one to one (Hungarian) at IoU >= 0.7; a door counts when both its rooms match and its centre lies within 0.5 m of the true door. Plan score = room F1 x door F1.
- **Strict reward (the official score):** 16 clean plans form a gate, g = min(1, mean clean score / 0.6); reward = g x mean score over the 48 hard plans (16 per hard style).
- **Partial credit (dense RL signal):** mean over plans of (room F1 + door F1) / 2. The strict reward is near zero for most early programs; partial credit still moves when a program gets rooms right but misses doors, so the model sees progress.

## What generalizes for anyone using Beam on code-writing tasks
- Feedback beats a single try: graded retries took Beam from 0.000 (round 1) to 0.179 held-out partial credit with no weight change.
- Write down what worked: a 46-line tips playbook more than doubled the held-out score (0.032 -> 0.075, 3 repeats each).
- Grade on data the feedback never saw: 64 held-out plans from other buildings keep the gain honest.
- Score partial credit as well as strict: strict was 0.000 in every run, so only partial credit shows learning.

## Beam reading its failure report (reasoning_content, Beam low, 10-round run)
> **Round 5** (practice-plan partial credit 0.022 -> 0.179): "My previous approach using simple thresholding produced 0 rooms on scan styles and poor results. I need a more robust approach."

> **Round 7** (practice-plan partial credit 0.078 -> 0.174): "The previous attempt has low IoU due to morphological closing causing over-merge and room splitting. Let me focus on better wall detection and not over-engineer doors yet."

Full traces: `runs/beam-low-best10/round-5/reasoning.md`, `runs/beam-low-best10/round-7/reasoning.md`.

## Results
| Model | Rounds | Best public strict | Best public partial | Held-out strict (64 plans) | Held-out partial | Best round |
|---|---|---|---|---|---|---|
| Beam low effort, 3 rounds (repeat 1) | 3 | 0.000 | 0.000 | 0.000 | 0.000 | round 1 |
| Beam low effort, 3 rounds (repeat 2) | 3 | 0.000 | 0.026 | 0.000 | 0.035 | round 3 |
| Beam low effort, 3 rounds (repeat 3) | 3 | 0.000 | 0.058 | 0.000 | 0.061 | round 2 |
| Beam low + playbook, 3 rounds (repeat 1) | 3 | 0.000 | 0.101 | 0.000 | 0.102 | round 1 |
| Beam low + playbook, 3 rounds (repeat 2) | 3 | 0.000 | 0.000 | 0.000 | 0.000 | round 1 |
| Beam low + playbook, 3 rounds (repeat 3) | 3 | 0.000 | 0.136 | 0.000 | 0.123 | round 1 |
| Beam high effort, 3 rounds | 3 | 0.000 | 0.010 | 0.000 | 0.016 | round 2 |
| Beam high + playbook, 3 rounds | 3 | 0.000 | 0.000 | 0.000 | 0.000 | round 1 |
| Beam low, 10 rounds, each revises the best code so far | 10 | 0.000 | 0.182 | 0.000 | 0.179 | round 9 |
| Beam high, 5 rounds, best of 4 samples per round | 5 | 0.000 | 0.181 | 0.000 | 0.176 | round 4 |
| Earlier baseline: Opus 5.5, one 2-hour agent run with tools (a different setup; strict score) | - | - | - | 0.032 | - | - |
| Reference solution (hand-written, for scale) | - | - | - | 0.242 | 0.586 | - |

Strict = the official score (below). Partial = the dense RL signal. Public scores drive the feedback; the 64 held-out plans are used only once, for the final score. Per-round scores, plan overlays and the wheelchair route are on the Space.

## Layout
| Path | What |
|---|---|
| `task/instruction.md` | the task statement given to the model |
| `task/workspace/README.md` | full input/output contract |
| `task/workspace/docs/STYLE-GUIDE.md` | drawing styles, door conventions, ranges |
| `task/workspace/public/` | 24 public plans (6 per style): `cases/pNN/plan.png` + `manifest.json`, answers in `answers/pNN.json` |
| `task/workspace/selfcheck/` | self-check, the exact run rules (`runner.py`) and scorer (`score.py`) |
| `task/workspace/submission/run.py` | starting stub (scores 0) |
| `task/tests/` | held-out grader: `verify.py`, `score.py`, the renderer `gen/`, the 64 held-out apartments (`hidden/units/`, geometry), `hidden/index.json`, the render seed and `hidden/expected.json` (hashes of every rendered plan and answer). The verifier renders the 64 held-out plans and their answers from these at grade time. |
| `task/hidden/` | the 64 held-out plans, rendered: `hNN/plan.png`, `manifest.json`, `answer.json` (same format as the public answers), `index.json` (style, scale, counts) |
| `task/environment/` | Docker setup of the sandbox the program runs in (Ubuntu 24.04, Python 3.12, pinned packages) |
| `code/env.py` | `FloorplanEnv`: `reset()` returns the task prompt; `step(code)` returns (reward, partial, feedback, info) |
| `code/grade.py` | grades a `run.py` with the Docker images (public self-check or held-out verifier); prints strict reward, partial credit, per-style F1, failures |
| `code/llm.py`, `code/loop.py` | the model loop (Beam via the Reflection API, Kimi K3) |
| `code/build_space.py` | builds the Space page: table, chart, overlays, networkx wheelchair route |
| `runs/<model>/round-<k>/` | every round: `prompt.md`, `response.md`, `reasoning.md` (Beam's thinking), `run.py`, `public.json` (scores); `runs/<model>/hidden.json` final held-out score |

## Use it for RL
```python
from code.env import FloorplanEnv, extract_code   # run from the repo root; grading needs the two Docker images (How to run)
env = FloorplanEnv(set="public")                  # 24 practice plans; set="hidden" only for the final score
prompt = env.reset()                              # instruction + contract + style guide + answer format
reward, partial, feedback, info = env.step(extract_code(model_reply))   # strict reward, dense reward, per-style report
next_prompt = prompt + "\n\nYour previous run.py:\n" + code + "\n\nGrader feedback:\n" + feedback
```
A GRPO-style reward function (for example TRL's `GRPOTrainer(reward_funcs=[floorplan_reward])`), using the dense partial credit so that early programs still get a signal:
```python
def floorplan_reward(completions, **kwargs):
    out = []
    for c in completions:
        text = c if isinstance(c, str) else c[-1]["content"]
        reward, partial, feedback, info = FloorplanEnv(set="public").step(extract_code(text))
        out.append(partial + reward)   # strict reward adds on top once door F1 > 0
    return out
```
The task is far from saturated: the strict reward is still 0.000 for every model loop here, while a hand-written reference reaches 0.242. There is a lot of room to train.

## Why trust these numbers
The grader is exact (rooms at IoU 0.7, doors within 0.5 m). Feedback uses only the 24 practice plans; the 64 held-out plans are from other buildings and never feed back. Every prompt, reasoning trace, program and score is in `runs/`.

**Known limits.** Kimi K3 did not finish (no reply to its round-1 request in 31 min); Beam high effort finished (held-out 0.016 in a single 3-round run; 0.176 with best of 4 samples per round). Most numbers are one run; the playbook result has 3 repeats with a wide spread. Doors are nearly unsolved (door F1 0.000 on every setup but one). We did not update Beam's weights.

## How to run
```bash
# 1. Build the sandbox and the held-out grader (Docker, linux/amd64)
docker build -t fp-agent task/environment
docker build -t fp-verifier --build-arg TASK_IMAGE=fp-agent task/tests

# 2. Public self-check of your program (24 plans, official runner and scorer)
docker run --rm -v "$PWD/my_submission:/mnt/sub:ro" --entrypoint bash fp-verifier -c \
  'rm -rf /app/submission/* && cp -a /mnt/sub/. /app/submission/ && python3 /app/selfcheck/selfcheck.py'

# 3. Held-out score (renders the 64 held-out plans, runs your program as an unprivileged user, writes /logs/verifier/reward.json)
docker run --rm -v "$PWD/my_submission:/mnt/sub:ro" --entrypoint bash fp-verifier -c \
  'rm -rf /app/submission/* && cp -a /mnt/sub/. /app/submission/ && /tests/test.sh; cat /logs/verifier/reward.json'

# 4. The RL loop (Beam writes run.py, gets scores and failures, retries); needs REFLECTION_API_KEY
#    code/grade.py runs the docker commands above over ssh on FP_HOST (a Docker host with the two images)
python3 code/loop.py --model beam --effort high
```
Without Docker: `cp my_run.py task/workspace/submission/run.py && python3 task/workspace/selfcheck/selfcheck.py` (needs Python 3.12 with numpy, scipy, opencv, scikit-image, shapely, networkx, Pillow).

## Credit and license
Plan geometry: **Swiss Dwellings v3.0.0** (Standfest et al.), Zenodo, DOI [10.5281/zenodo.7788422](https://doi.org/10.5281/zenodo.7788422), CC BY 4.0.
The plans here are re-rendered from that geometry. This repo is released under CC BY 4.0.
Built by Hamood (hamoodey) for Reflection's "Build with Beam" contest.