Add perception ablation grid (observation channel × fog of war)

`fog_mode` now spans a 3×2 grid — three observation channels, each
fogged or `-clear` (no fog):

- structured — text briefing + a text 'unexplored regions' block.
- vision — text briefing + PNG minimap (the text already lists
units/enemies, so the image is a supplement).
- image — image-PRIMARY: the text briefing is redacted of every
coordinate; a labelled minimap (per-unit handles,
greedy collision avoidance + leader lines) is the sole
spatial source. The clean image-reading probe. The
model references units by the on-map handle (tank-1,
enemy-2); the agent maps it back to the engine id.

The `-clear` variants disable fog via the engine `reveal_map` flag —
perfect-information controls that isolate the perception cost; the
no-cheat bar applies only to the fogged cells.

- schema.py: PERCEPTION_MODES + CompiledLevel.reveal_map/obs_channel.
- eval_core.py: emit `reveal_map: true` into the engine scenario YAML
for `-clear` cells.
- prompt_v2.py: perception_labels (stable per-actor handles) +
briefing_image_primary (position-redacted briefing).
- minimap.py: render_tactical_minimap unit_labels — legible per-unit
labels with collision avoidance and leader lines.
- agent.py: image-primary channel dispatch, handle↔engine-id
translation, string-typed tool schemas for the image channel.
- run_eval.py: --perception-sweep expands every pack:level into the
six modality cells (pack:level:<mode>).

Pinned by tests/test_perception_ablation.py.

CLAUDE.md

@@ -195,6 +195,39 @@ A scenario is defective if any of the following hold:
   `stance:3` AGENT combat unit auto-hunts the whole map; for a
   perception pack keep the agent's combat arm `stance:0` so a
   stall policy can't win for free by self-delivering the army.
+- **`reveal_map:` — the no-fog perception cell** (pinned by
+  `OpenRA-Rust/openra-data/tests/test_reveal_map.rs` +
+  `tests/test_perception_ablation.py`). A scenario may declare a
+  top-level `reveal_map: true`; the agent player then observes the
+  whole map with NO fog of war — every enemy actor is reported
+  regardless of shroud and `explored_percent` is 100. Parsed by
+  `oramap.rs::parse_scenario_yaml` (mirrors `spawn_mcvs`), applied
+  in `env.rs` (`is_visible_to` short-circuits true for the agent;
+  `refresh_explored_cells` fills the playable rectangle). This is
+  the no-fog half of the **perception ablation grid**. The bench
+  `fog_mode` spans 3 observation channels × 2 fog states = 6 cells
+  (`openra_bench/scenarios/schema.py::PERCEPTION_MODES`):
+  - `structured` — text briefing + a text 'Unexplored regions'
+    block; no image.
+  - `vision` — text briefing + PNG minimap. NOTE the briefing
+    already enumerates units/enemies with coordinates, so the image
+    is a redundant SUPPLEMENT — `vision` does NOT isolate
+    image-reading.
+  - `image` — image-PRIMARY: `prompt_v2.briefing_image_primary`
+    redacts every coordinate from the text; the labelled minimap
+    (`render_tactical_minimap(..., unit_labels=...)`) is the sole
+    spatial source. The model references units by the on-map
+    handle (`tank-1`, `enemy-2`); `agent._to_commands` maps the
+    handle back to the engine id. The clean "can the model read a
+    minimap" probe.
+  Each channel has a fogged form and a `-clear` (no-fog) form. A
+  `-clear` cell is a perfect-information CONTROL that isolates the
+  perception cost — a stall/observe policy WINS a `-clear`
+  perception pack (perception removed), so the no-cheat bar
+  applies ONLY to the fogged cells, never the `-clear` ones. Run
+  the full grid with `run_eval --perception-sweep` (expands every
+  `pack:level` into `pack:level:<mode>`); the human Play tab stays
+  on the canonical `vision` (fogged) modality.
 - **`pbox` costs 600** (not the 400 some old specs assumed);
   defense and infantry are SEPARATE production queues so an
   efficient policy queues `build('pbox')` and `build('e1')` in

openra_bench/agent.py

@@ -375,39 +375,50 @@ def _render_minimap_b64(
         return None
 
 
-def _to_commands(tool_calls: list[dict], Command: Any) -> list:
+def _to_commands(
+    tool_calls: list[dict], Command: Any, label_to_id: dict | None = None
+) -> list:
+    # In the image-primary channel the model references units by the
+    # legible handle shown on the minimap (`tank-1`); map it back to the
+    # engine actor id. Numeric ids (every other channel) pass straight
+    # through — the lookup simply misses.
+    label_to_id = label_to_id or {}
+
+    def _rid(x: Any) -> str:
+        return label_to_id.get(str(x), str(x))
+
     cmds = []
     for call in tool_calls:
         name = _TOOL_ALIASES.get(call.get("name", ""), call.get("name", ""))
         args = call.get("arguments") or {}
         try:
             if name == "move_units":
-                ids = [str(i) for i in args["unit_ids"]]
+                ids = [_rid(i) for i in args["unit_ids"]]
                 cmds.append(
                     Command.move_units(ids, int(args["target_x"]), int(args["target_y"]))
                 )
             elif name == "attack_unit":
-                ids = [str(i) for i in args["unit_ids"]]
-                cmds.append(Command.attack_unit(ids, str(args["target_id"])))
+                ids = [_rid(i) for i in args["unit_ids"]]
+                cmds.append(Command.attack_unit(ids, _rid(args["target_id"])))
             elif name == "guard":
-                ids = [str(i) for i in args["unit_ids"]]
-                cmds.append(Command.guard(ids, str(args["target_id"])))
+                ids = [_rid(i) for i in args["unit_ids"]]
+                cmds.append(Command.guard(ids, _rid(args["target_id"])))
             elif name == "enter_transport":
-                ids = [str(i) for i in args["unit_ids"]]
+                ids = [_rid(i) for i in args["unit_ids"]]
                 cmds.append(
-                    Command.enter_transport(ids, str(args["target_id"]))
+                    Command.enter_transport(ids, _rid(args["target_id"]))
                 )
             elif name == "observe":
                 cmds.append(Command.observe())
             elif name == "surrender":
                 cmds.append(Command.surrender())
             elif name == "set_stance":
-                ids = [str(i) for i in args["unit_ids"]]
+                ids = [_rid(i) for i in args["unit_ids"]]
                 cmds.append(Command.set_stance(ids, int(args["stance"])))
             elif name == "patrol":
-                cmds.append(Command.patrol([str(i) for i in args["unit_ids"]]))
+                cmds.append(Command.patrol([_rid(i) for i in args["unit_ids"]]))
             elif name in ("attack_move", "harvest", "set_rally_point"):
-                ids = [str(i) for i in args["unit_ids"]]
+                ids = [_rid(i) for i in args["unit_ids"]]
                 fn = getattr(Command, name)
                 cmds.append(fn(ids, int(args["target_x"]), int(args["target_y"])))
             elif name in (
@@ -419,7 +430,7 @@ def _to_commands(tool_calls: list[dict], Command: Any) -> list:
                 "set_primary",
                 "unload",
             ):
-                ids = [str(i) for i in args["unit_ids"]]
+                ids = [_rid(i) for i in args["unit_ids"]]
                 cmds.append(getattr(Command, name)(ids))
             elif name in ("build", "cancel_production"):
                 cmds.append(getattr(Command, name)(str(args["item"])))
@@ -434,6 +445,35 @@ def _to_commands(tool_calls: list[dict], Command: Any) -> list:
     return cmds
 
 
+def _image_primary_tools(tools: list[dict]) -> list[dict]:
+    """Re-type unit/target handles as strings for the image-primary
+    channel: the model references actors by the legible label drawn on
+    the minimap (`tank-1`, `enemy-2`), not numeric engine ids. The
+    `_to_commands` `label_to_id` map turns them back into engine ids."""
+    import copy
+
+    out = copy.deepcopy(tools)
+    for t in out:
+        props = (
+            t.get("function", {}).get("parameters", {}).get("properties", {})
+        )
+        ui = props.get("unit_ids")
+        if isinstance(ui, dict) and ui.get("type") == "array":
+            ui["items"] = {"type": "string"}
+            ui["description"] = (
+                'unit handles EXACTLY as labelled on the minimap, '
+                'e.g. ["tank-1","jeep-2"]'
+            )
+        tid = props.get("target_id")
+        if isinstance(tid, dict):
+            tid["type"] = "string"
+            tid["description"] = (
+                'the target actor\'s handle as labelled on the minimap, '
+                'e.g. "enemy-1"'
+            )
+    return out
+
+
 class ModelAgent:
     """One instance per episode (keeps bounded chat history).
 
@@ -462,6 +502,15 @@ class ModelAgent:
         self._level = level
         # Scenario config wins over the model-side cfg default.
         self._fog_mode = fog_mode or getattr(cfg, "fog_mode", "vision")
+        # Image-primary channel: the text briefing carries no positions —
+        # the labelled minimap is the sole spatial source, and the model
+        # references units by those labels. Re-type the tool handles to
+        # strings; `_labels` / `_label_to_id` are rebuilt each turn.
+        self._image_primary = self._fog_mode.startswith("image")
+        if self._image_primary:
+            self.tools = _image_primary_tools(self.tools)
+        self._labels: dict[str, str] = {}
+        self._label_to_id: dict[str, str] = {}
         # Real terrain (map.png from the .oramap) for the vendored
         # training bitmap minimap; persistent fog history across turns.
         self._terrain: bytes | None = None
@@ -484,6 +533,16 @@ class ModelAgent:
             sys_content = SYSTEM_PROMPT + (
                 f"\n\n{system_extra}" if system_extra else ""
             )
+        if self._image_primary:
+            sys_content += (
+                "\n\nPERCEPTION MODE — IMAGE-PRIMARY. The text briefing "
+                "lists WHAT units exist but never where anything is. "
+                "Every position — your units AND the enemy — is shown "
+                "ONLY on the minimap image. Each marker is tagged with a "
+                "legible label (tank-1, jeep-2, enemy-1). Read the image "
+                "to locate units and threats; pass those exact labels as "
+                "the ids in your tool calls (e.g. unit_ids=[\"tank-1\"])."
+            )
         self.history: list[dict] = [{"role": "system", "content": sys_content}]
         self.stats = {"turns": 0, "tool_calls": 0, "empty_replies": 0}
         # Controller contract (openra_bench/controller.py): a ModelAgent
@@ -492,7 +551,60 @@ class ModelAgent:
         # all drive it interchangeably with any other policy backend.
         self.name = getattr(cfg, "model", None) or "model"
 
+    def _image_primary_message(self, render_state: dict) -> dict:
+        """Image-primary turn message: a position-redacted text briefing
+        plus a labelled minimap — the minimap is the ONLY place the
+        model learns where its units and the enemy are."""
+        from .prompt_v2 import briefing_image_primary, perception_labels
+
+        # Carry last turn's map forward so a label stays pinned to its
+        # actor for the whole episode (stable handles across turns).
+        self._labels = perception_labels(render_state, self._labels)
+        self._label_to_id = {v: k for k, v in self._labels.items()}
+        text = briefing_image_primary(render_state, self._labels)
+        b64 = None
+        try:
+            import base64
+            import io
+
+            from .minimap import render_tactical_minimap
+
+            # Keep the PNG ≤ ~1560px wide so the vision API does not
+            # downscale it (which would shrink the unit labels below
+            # legibility); the 6px base cell × scale sets the width.
+            rows = [
+                r for r in (render_state.get("minimap") or "").split("\n")
+                if r
+            ]
+            w = max((len(r) for r in rows), default=64)
+            scale = max(2, min(6, 1560 // max(1, w * 6)))
+            img = render_tactical_minimap(
+                render_state, scale=scale, unit_labels=self._labels,
+            )
+            if img is not None:
+                buf = io.BytesIO()
+                img.save(buf, "PNG")
+                b64 = base64.b64encode(buf.getvalue()).decode("ascii")
+        except Exception:  # noqa: BLE001 — degrade to text-only on render fail
+            b64 = None
+        if b64:
+            return {
+                "role": "user",
+                "content": [
+                    {"type": "text", "text": text},
+                    {
+                        "type": "image_url",
+                        "image_url": {"url": f"data:image/png;base64,{b64}"},
+                    },
+                ],
+            }
+        return {"role": "user", "content": text}
+
     def _user_message(self, render_state: dict) -> dict:
+        # Image-primary channel builds its own (position-redacted)
+        # briefing + labelled minimap — dispatch before the text path.
+        if self._image_primary:
+            return self._image_primary_message(render_state)
         # Briefing = vendored training briefing_v2 (one unit/line,
         # "moving to (x,y)", Idle list). Objective is in the system
         # prompt now, so it's NOT repeated here.
@@ -502,10 +614,11 @@ class ModelAgent:
             text = _v2_brief(render_state)
         except Exception:  # noqa: BLE001 — never break a turn
             text = build_briefing(render_state, self.objective)
-        # Structured-fog mode: NO image — append the text "Unexplored
-        # regions" block instead (text-vs-vision A/B; pair with the
-        # easy/medium level of the setup).
-        if self._fog_mode == "structured":
+        # Structured channel: NO image — append the text "Unexplored
+        # regions" block instead (text-vs-vision A/B). Covers both
+        # `structured` (fogged) and `structured-clear` (no fog — under
+        # reveal_map the block reports zero unexplored regions).
+        if self._fog_mode.startswith("structured"):
             try:
                 from .prompt_v2 import structured_fog as _v2_fog
 
@@ -608,7 +721,7 @@ class ModelAgent:
                 ],
             }
         )
-        cmds = _to_commands(reply.tool_calls, Command)
+        cmds = _to_commands(reply.tool_calls, Command, self._label_to_id)
         self.stats["tool_calls"] += len(cmds)
         if not cmds:
             self.stats["empty_replies"] += 1

openra_bench/eval_core.py

@@ -67,6 +67,11 @@ def _scenario_to_tmp_yaml(compiled: CompiledLevel) -> str:
     sched = getattr(compiled, "scheduled_events", None) or []
     if sched:
         data["scheduled_events"] = sched
+    # No-fog perception cells (fog_mode ends with "-clear") flip the
+    # engine's `reveal_map` flag: the agent observes the whole map with
+    # no fog of war — the clear half of the perception ablation grid.
+    if getattr(compiled, "reveal_map", False):
+        data["reveal_map"] = True
     fd = tempfile.NamedTemporaryFile(
         "w", suffix=f"_{compiled.pack_id}_{compiled.level}.yaml", delete=False
     )

openra_bench/minimap.py

@@ -284,6 +284,7 @@ def render_tactical_minimap(
     legend: bool = True,
     selected=None,
     arrows=None,
+    unit_labels=None,
 ):
     """A legible tactical minimap as a PIL RGB image:
 
@@ -295,7 +296,13 @@ def render_tactical_minimap(
       strip beneath the map.
 
     `scale` multiplies the 6px base cell. Returns None if Pillow is
-    missing or there is nothing to draw."""
+    missing or there is nothing to draw.
+
+    `unit_labels` (id-str → handle, e.g. `{"1004": "tank-1"}`) drives
+    the image-primary perception channel: every actor is tagged with
+    its legible handle so the model can identify and command units
+    from the picture alone (the text briefing carries no positions).
+    When set, the per-cell count badge is replaced by the labels."""
     try:
         from PIL import Image, ImageDraw
     except Exception:  # noqa: BLE001
@@ -341,7 +348,9 @@ def render_tactical_minimap(
             is_b = force_building or bool(it.get("is_building"))
             atype = (it.get("actor_type") or it.get("type") or "?")
             shape = _unit_shape(atype, is_b)
-            by_cell.setdefault((cx, cy), []).append((side, shape))
+            by_cell.setdefault((cx, cy), []).append(
+                (side, shape, it.get("id"))
+            )
             if side == "own" and atype != "?":
                 own_types.setdefault(str(atype).lower(), shape)
 
@@ -362,11 +371,14 @@ def render_tactical_minimap(
     badge_font = _minimap_font(max(9, int(cp * 0.62)))
     for (cx, cy), occ in by_cell.items():
         # Dominant occupant decides the shape; prefer a building.
-        side, shape = next(
+        side, shape, _aid = next(
             (o for o in occ if _is_bld(o[1])), occ[0]
         )
         _draw_unit_shape(draw, cx, cy, cp, shape, _color(side, shape))
-        if len(occ) > 1:
+        # The count badge and per-unit labels are mutually exclusive —
+        # `unit_labels` (image-primary) names each occupant individually,
+        # which already disambiguates a stack.
+        if len(occ) > 1 and not unit_labels:
             tx, ty = (cx + 1) * cp - cp * 0.42, cy * cp + 1
             draw.text(
                 (tx, ty), str(len(occ)), fill=(255, 255, 255),
@@ -426,6 +438,65 @@ def render_tactical_minimap(
                     stroke_width=3, stroke_fill=(0, 0, 0),
                 )
 
+    # Per-unit ID labels — the image-primary channel. Each actor's
+    # legible handle (`tank-1`, `enemy-2`) is placed near its marker
+    # with greedy collision avoidance (labels nudge clear of one
+    # another into free space) and a leader line when a label drifts
+    # off its cell — so the model can identify every unit from the
+    # picture alone. Drawn last so nothing occludes the text.
+    if unit_labels:
+        lab_font = _minimap_font(max(15, int(cp * 0.95)))
+        img_w, img_h = w * cp, h * cp
+        # One entry per actor id — an actor can appear in both the unit
+        # and building lists; dedup so its label is drawn once.
+        seen: set = set()
+        actors = []  # (cx, cy, label, side)
+        for (cx, cy), occ in by_cell.items():
+            for side, _shape, aid in occ:
+                key = str(aid)
+                if aid is None or key in seen or key not in unit_labels:
+                    continue
+                seen.add(key)
+                actors.append((cx, cy, unit_labels[key], side))
+        placed: list = []  # occupied label rects (x0, y0, x1, y1)
+
+        def _free(x0, y0, x1, y1):
+            return all(
+                x1 < r[0] or x0 > r[2] or y1 < r[1] or y0 > r[3]
+                for r in placed
+            )
+
+        for cx, cy, lab, side in sorted(actors, key=lambda a: (a[1], a[0])):
+            try:
+                bb = draw.textbbox((0, 0), lab, font=lab_font, stroke_width=3)
+                tw, th = bb[2] - bb[0], bb[3] - bb[1]
+            except Exception:  # noqa: BLE001
+                tw, th = len(lab) * 9, 16
+            mx, my = cx * cp + cp // 2, cy * cp + cp // 2
+            lx = min(max(0, mx + 4), img_w - tw - 1)
+            ly = my - cp - th
+            step = th + 3
+            for _ in range(60):  # nudge upward into free space
+                cand = min(max(1, ly), img_h - th - 1)
+                if _free(lx, cand, lx + tw, cand + th):
+                    ly = cand
+                    break
+                ly -= step
+            else:
+                ly = min(max(1, my + cp), img_h - th - 1)
+            placed.append((lx, ly, lx + tw, ly + th))
+            # Leader line when the label sits away from its marker.
+            if abs(lx + tw // 2 - mx) > cp or abs(ly + th // 2 - my) > cp:
+                draw.line(
+                    [(mx, my), (lx + 2, ly + th // 2)],
+                    fill=(165, 167, 178), width=1,
+                )
+            col = (175, 255, 175) if side == "own" else (255, 190, 172)
+            draw.text(
+                (lx, ly), lab, fill=col, font=lab_font,
+                stroke_width=3, stroke_fill=(0, 0, 0),
+            )
+
     # Legend strip — the unit TYPES actually present, each with its
     # own shape, so the player can read 1tnk vs 2tnk vs e3 etc.
     if legend:
@@ -449,10 +520,16 @@ def render_tactical_minimap(
             except Exception:  # noqa: BLE001
                 tw = len(tname) * cp * 0.5
             x = int(tx + tw + cp * 0.7)
-        draw.text(
-            (int(cp * 0.4), ly + int(cp * 1.05)),
+        help_txt = (
+            "green = you   red/orange = enemy   "
+            "label = unit id (pass it to your tools)   "
+            "scout the dark area to reveal more"
+            if unit_labels else
             "green = you   red/orange = enemy   number = units stacked  "
-            " white box = selected   arrow = move/attack order",
+            " white box = selected   arrow = move/attack order"
+        )
+        draw.text(
+            (int(cp * 0.4), ly + int(cp * 1.05)), help_txt,
             fill=(200, 202, 212), font=lfont,
         )
 

openra_bench/prompt_v2.py

@@ -191,6 +191,121 @@ def structured_fog(render_state: dict) -> str:
     return format_structured_fog(obs, bounds)
 
 
+# ── Image-primary channel ────────────────────────────────────────────
+# The `image` perception modality: the text briefing is redacted of
+# every coordinate, so the PNG minimap (with per-unit labels) is the
+# ONLY source of spatial state. Units carry a short legible handle
+# (`tank-1`, `enemy-2`) shown identically in the briefing roster, on
+# the minimap, and as the id the model passes to its tools.
+_FRIENDLY_TYPE = {
+    "1tnk": "tank", "2tnk": "tank", "3tnk": "tank", "4tnk": "tank",
+    "jeep": "jeep", "apc": "apc", "arty": "arty", "harv": "harvester",
+    "mcv": "mcv", "e1": "rifle", "e2": "grenadier", "e3": "rocket",
+    "e6": "engineer",
+}
+
+
+def _friendly_word(actor_type: object) -> str:
+    """A short legible word for an actor type — `1tnk` → `tank`. Falls
+    back to the raw type (so buildings read `proc-1`, `pbox-1`)."""
+    t = str(actor_type or "").lower()
+    return _FRIENDLY_TYPE.get(t, t or "unit")
+
+
+def perception_labels(
+    render_state: dict, prior: dict[str, str] | None = None
+) -> dict[str, str]:
+    """Per-actor handle map for the image-primary channel: engine id
+    (str) → a short legible label. Own actors get a type-word handle
+    (`tank-1`, `jeep-2`, `proc-1`); enemy actors get `enemy-N`.
+
+    `prior` (the previous turn's map) is carried forward so a label
+    stays pinned to its actor for the actor's whole lifetime — an
+    enemy revealed mid-episode keeps the same handle on every later
+    turn instead of renumbering when a lower-id foe is later scouted.
+    New actors are assigned in engine-id order, continuing the count."""
+    labels: dict[str, str] = dict(prior or {})
+    # Highest index already used per prefix — so new handles continue
+    # the sequence rather than colliding with carried-over ones.
+    used: dict[str, int] = {}
+    for lab in labels.values():
+        pre, _, num = lab.rpartition("-")
+        if pre and num.isdigit():
+            used[pre] = max(used.get(pre, 0), int(num))
+
+    def _assign(items, prefix_fn):
+        for a in sorted(items, key=lambda a: int(a.get("id", 0) or 0)):
+            aid = a.get("id")
+            if aid is None or str(aid) in labels:
+                continue
+            pre = prefix_fn(a)
+            used[pre] = used.get(pre, 0) + 1
+            labels[str(aid)] = f"{pre}-{used[pre]}"
+
+    _assign(
+        render_state.get("units_summary", []) or [],
+        lambda a: _friendly_word(a.get("type")),
+    )
+    _assign(
+        render_state.get("own_buildings", []) or [],
+        lambda a: _friendly_word(a.get("type")),
+    )
+    enemies = list(render_state.get("enemy_summary", []) or [])
+    enemies += list(render_state.get("enemy_buildings_summary", []) or [])
+    _assign(enemies, lambda a: "enemy")
+    return labels
+
+
+def briefing_image_primary(render_state: dict, labels: dict[str, str]) -> str:
+    """Image-primary briefing: every coordinate redacted. The text
+    keeps the non-spatial scaffolding (funds, power, the unit roster as
+    label+type handles, production) so the model knows WHAT it has —
+    the minimap image is the only place it learns WHERE anything is."""
+    st = state_from_render(render_state)
+    econ = st["economy"]
+    funds = econ["cash"] + econ["ore"]
+    out = [
+        f"--- TURN BRIEFING (tick {st['tick']}) ---",
+        f"Funds: ${funds} (cash=${econ['cash']} + ore=${econ['ore']}) | "
+        f"Power: {st['power_balance']:+d} | "
+        f"Harvesters: {econ['harvester_count']} | "
+        f"Explored: {st['explored_percent']}%",
+    ]
+    units = sorted(
+        st["units_summary"], key=lambda u: int(u.get("id", 0) or 0)
+    )
+    out.append(f"Your units ({len(units)}) — find each on the minimap:")
+    idle = []
+    for u in units:
+        lab = labels.get(str(u.get("id")), str(u.get("id")))
+        out.append(f"  {lab}  ({u.get('type') or '?'})")
+        if str(u.get("activity", "")).lower() in ("", "idle"):
+            idle.append(lab)
+    if idle:
+        out.append(f"Idle: {', '.join(idle)}")
+    bsum = st["buildings_summary"]
+    if bsum:
+        out.append(f"Your buildings ({len(bsum)}) — find each on the minimap:")
+        for b in bsum:
+            lab = labels.get(str(b.get("id")), str(b.get("id")))
+            out.append(f"  {lab}  ({b.get('type') or '?'})")
+    else:
+        out.append("Buildings: none (you command mobile units only).")
+    out.append(
+        "Enemies: NOT listed here — read the minimap. Enemy markers are "
+        "labelled enemy-1, enemy-2, … (only those in your units' sight; "
+        "scout the fog to reveal more)."
+    )
+    prod = st["production_items"]
+    out.append(f"Production: {', '.join(prod) if prod else 'IDLE'}")
+    out.append(
+        "ALL unit and enemy POSITIONS are on the minimap image only — "
+        "each marker is labelled with the id shown above; pass that id "
+        "to your tools."
+    )
+    return "\n".join(out)
+
+
 def minimap_b64(
     render_state: dict, terrain_png: bytes | None,
     explored_history: set | None,

openra_bench/run_eval.py

@@ -117,12 +117,18 @@ def evaluate(
     dry_run: bool = False,
     report_path: str | Path | None = None,
     progress=None,
+    perception_sweep: bool = False,
 ) -> dict:
     """Run packs×levels×seeds. If `held_out_seeds` is given, those are
     run too and tagged split='held_out'; the report adds
     `overall_held_out` and `generalization_gap` (public composite −
     held-out composite) — the anti-memorization metric the
     generalization literature (Procgen/SMACv2/lmgame-Bench) requires.
+
+    `perception_sweep` expands every pack×level into the 4 perception
+    ablation cells (`pack:level:<mode>` for mode in PERCEPTION_MODES —
+    vision/structured × fog/no-fog) instead of the raw 3 levels, so one
+    run yields the full channel-cost / fog-cost decomposition.
     """
     from .resilience import (
         BudgetExceeded,
@@ -201,9 +207,22 @@ def evaluate(
                 f"{pack.meta.quarantine_reason or 'excluded from default set'})"
             )
             continue
+        # Perception sweep: every level × the 4 modality cells
+        # (pack:level:<mode>). Overrides both declared configs and the
+        # raw enumeration — it is an explicit ablation request.
+        if perception_sweep:
+            from .scenarios.schema import PERCEPTION_MODES
+
+            unit_iter = []
+            for lv in levels:
+                for mode in PERCEPTION_MODES:
+                    cl = compile_level(pack, lv)
+                    cl.fog_mode = mode
+                    cl.config_name = f"{lv}:{mode}"
+                    unit_iter.append((cl, f"{pack.meta.id}:{lv}:{mode}"))
         # Declared configs (pack:config_name, each pins level+fog_mode)
         # supersede the raw 3-level enumeration when present.
-        if pack.configs:
+        elif pack.configs:
             from .scenarios.loader import is_map_supported
 
             ms = is_map_supported(pack.base_map)
@@ -577,6 +596,10 @@ def main(argv: list[str]) -> int:
     ap.add_argument("--fog-mode", default="vision",
                     choices=["vision", "structured"],
                     help="spatial channel: PNG minimap vs text fog")
+    ap.add_argument("--perception-sweep", action="store_true",
+                    help="run the 2x2 perception ablation: every "
+                    "pack:level expanded into vision/structured x "
+                    "fog/no-fog (pack:level:<mode>)")
     a = ap.parse_args(argv[1:])
 
     cfg = None
@@ -618,6 +641,7 @@ def main(argv: list[str]) -> int:
         smoke=a.smoke,
         dry_run=a.dry_run,
         report_path=a.out,
+        perception_sweep=a.perception_sweep,
         progress=lambda d, n, rec, c: print(
             f"[{d}/{n}] {rec['cell']}:{rec['split']}#{rec['seed']} "
             f"{rec['outcome']} comp={rec['composite']} "

openra_bench/scenarios/schema.py

@@ -118,16 +118,47 @@ class Level(BaseModel):
     forbidden_tools: list[str] = Field(default_factory=list)
 
 
+# The perception ablation grid: observation channel × fog of war.
+#
+# Three channels:
+#   structured — text briefing + a text 'Unexplored regions' block;
+#                NO image. The text-only condition.
+#   vision     — text briefing + PNG minimap. The multimodal "both
+#                available" condition — note the text briefing already
+#                enumerates units/enemies, so the image is a SUPPLEMENT.
+#   image      — image-PRIMARY: the text briefing is redacted of every
+#                coordinate (and the enemy line dropped); the PNG, with
+#                legible per-unit labels, is the ONLY source of spatial
+#                state. The clean "can the model read a minimap" probe.
+#
+# Fog axis: bare name ⇒ fog ON (canonical scoring modality); the
+# `-clear` variant reveals the whole map (engine `reveal_map: true` ⇒
+# every enemy observed, `explored_percent` 100). A `-clear` cell is a
+# perfect-information CONTROL for measuring the perception cost — the
+# no-cheat bar applies only to the fogged cells.
+PERCEPTION_MODES = (
+    "structured", "structured-clear",
+    "vision", "vision-clear",
+    "image", "image-clear",
+)
+FogMode = Literal[
+    "structured", "structured-clear",
+    "vision", "vision-clear",
+    "image", "image-clear",
+]
+
+
 class ScenarioConfig(BaseModel):
     """A named runnable configuration of one pack: pins a difficulty
-    `level` and the observation `fog_mode`. The same setup at
-    `fog_mode: structured` (text 'Unexplored regions') vs `vision`
-    (PNG minimap) becomes two distinct cells, so text-vs-vision is a
-    first-class comparison the YAML declares (not just a CLI flag)."""
+    `level` and the observation `fog_mode`. `fog_mode` spans the 2×2
+    perception grid — channel (`vision` PNG minimap vs `structured`
+    text 'Unexplored regions') × fog (on, or `-clear` ⇒ no fog) — so
+    text-vs-vision AND fogged-vs-clear are first-class comparisons the
+    YAML declares, not just CLI flags."""
 
     name: str = Field(..., description="cell suffix, e.g. easy-structured")
     level: LevelName
-    fog_mode: Literal["vision", "structured"] = "vision"
+    fog_mode: FogMode = "vision"
     # Optional override of the level's objective_coords for this cell.
     objective_coords: Literal["exact", "relative"] | None = None
 
@@ -155,9 +186,9 @@ class CompiledLevel(BaseModel):
     map_supported: bool = Field(
         ..., description="False => Rust lacks this map (Phase 3 gate)"
     )
-    # Observation channel + the cell label. config_name is None for
-    # legacy level cells (pack:level); set for declared configs
-    # (pack:config_name).
+    # Observation modality (channel × fog — see PERCEPTION_MODES) + the
+    # cell label. config_name is None for legacy level cells
+    # (pack:level); set for declared configs / sweep cells.
     fog_mode: str = "vision"
     config_name: str | None = None
     objective_coords: Literal["exact", "relative"] = "exact"
@@ -171,6 +202,24 @@ class CompiledLevel(BaseModel):
     # scenario, and re-attached at YAML-write time.
     scheduled_events: list[dict[str, Any]] = Field(default_factory=list)
 
+    @property
+    def reveal_map(self) -> bool:
+        """No-fog cell? The `-clear` perception modes disable fog of
+        war — `_scenario_to_tmp_yaml` emits `reveal_map: true` so the
+        engine reveals the whole map to the agent."""
+        return self.fog_mode.endswith("-clear")
+
+    @property
+    def obs_channel(self) -> str:
+        """The observation channel — `structured` (text only), `image`
+        (image-primary: text redacted, PNG is the sole spatial source),
+        or `vision` (text + PNG) — independent of the fog axis."""
+        if self.fog_mode.startswith("structured"):
+            return "structured"
+        if self.fog_mode.startswith("image"):
+            return "image"
+        return "vision"
+
 
 class ScenarioPack(BaseModel):
     """The contributor-authored unit. One file = one decision problem."""

tests/test_perception_ablation.py

@@ -0,0 +1,205 @@
+"""The perception ablation grid: observation channel × fog of war.
+
+`fog_mode` spans 3 channels × 2 fog states = 6 cells:
+  structured / structured-clear  — text only, no image
+  vision     / vision-clear      — text briefing + PNG minimap
+  image      / image-clear       — image-PRIMARY: text redacted of all
+                                   positions; the labelled minimap is
+                                   the sole spatial source.
+The `-clear` cells disable fog (engine `reveal_map: true`) — they are
+perfect-information CONTROLS that isolate the perception cost; the
+fogged cells keep the no-cheat bar.
+
+Decomposition the grid enables:
+  score(vision-clear) − score(vision)     = partial-observability cost
+  score(vision)       − score(image)      = text-scaffolding crutch
+  score(vision)       − score(structured) = visual-channel advantage
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import pytest
+import yaml
+
+pytest.importorskip("openra_rl_training", reason="Rust env wheel not installed")
+
+from openra_bench.eval_core import _scenario_to_tmp_yaml, run_level
+from openra_bench.scenarios import load_pack
+from openra_bench.scenarios.loader import compile_level
+from openra_bench.scenarios.schema import PERCEPTION_MODES, ScenarioConfig
+
+PACKS = Path(__file__).parent.parent / "openra_bench" / "scenarios" / "packs"
+_PACK = "perception-count-the-threat.yaml"
+
+
+def test_perception_modes_are_the_3x2_grid():
+    assert set(PERCEPTION_MODES) == {
+        "structured", "structured-clear",
+        "vision", "vision-clear",
+        "image", "image-clear",
+    }
+
+
+@pytest.mark.parametrize("mode", PERCEPTION_MODES)
+def test_scenario_config_accepts_every_mode(mode):
+    cfg = ScenarioConfig(name="cell", level="easy", fog_mode=mode)
+    assert cfg.fog_mode == mode
+
+
+def test_compiled_level_reveal_map_and_channel_properties():
+    """`-clear` ⇒ reveal_map True; channel is independent of fog."""
+    c = compile_level(load_pack(PACKS / _PACK), "easy")
+    cases = {
+        "structured": (False, "structured"),
+        "structured-clear": (True, "structured"),
+        "vision": (False, "vision"),
+        "vision-clear": (True, "vision"),
+        "image": (False, "image"),
+        "image-clear": (True, "image"),
+    }
+    for mode, (reveal, channel) in cases.items():
+        c.fog_mode = mode
+        assert c.reveal_map is reveal, mode
+        assert c.obs_channel == channel, mode
+
+
+def test_tmp_yaml_emits_reveal_map_only_for_clear_modes():
+    """The no-fog cells must write `reveal_map: true` into the engine
+    scenario YAML; the fogged cells must not mention it at all."""
+    c = compile_level(load_pack(PACKS / _PACK), "easy")
+    for mode in PERCEPTION_MODES:
+        c.fog_mode = mode
+        data = yaml.safe_load(Path(_scenario_to_tmp_yaml(c)).read_text())
+        if mode.endswith("-clear"):
+            assert data.get("reveal_map") is True, mode
+        else:
+            assert "reveal_map" not in data, mode
+
+
+def test_no_fog_cell_reveals_what_the_fogged_cell_hides():
+    """End-to-end: an identical perception scenario, observe-only.
+    Under fog a stall sees nothing; under `reveal_map` every enemy is
+    observed and the whole map is explored — the perception control."""
+    pack = load_pack(PACKS / _PACK)
+    stall = lambda rs, Command: [Command.observe()]  # noqa: E731
+
+    fogged = compile_level(pack, "medium")
+    fogged.fog_mode = "vision"
+    f = run_level(fogged, stall, seed=1)
+
+    clear = compile_level(pack, "medium")
+    clear.fog_mode = "vision-clear"
+    c = run_level(clear, stall, seed=1)
+
+    # Fog hides the enemies from an observe-only policy.
+    assert len(f.signals.enemies_seen_ids) == 0
+    assert f.signals.explored_percent < 50.0
+    # No-fog reveals every enemy and the entire map.
+    assert len(c.signals.enemies_seen_ids) > 0
+    assert c.signals.explored_percent > 99.0
+
+
+def test_perception_sweep_expands_to_six_cells_per_level():
+    from openra_bench.run_eval import evaluate
+
+    out = evaluate(
+        [PACKS / _PACK], levels=["easy"], seeds=[1],
+        perception_sweep=True, dry_run=True,
+    )
+    expected = {
+        f"perception-count-the-threat:easy:{m}" for m in PERCEPTION_MODES
+    }
+    assert set(out["cells"]) == expected, out["cells"]
+
+
+# ── Image-primary channel ────────────────────────────────────────────
+
+def _render_state(mode: str) -> dict:
+    """A live render_state for the count-the-threat pack under `mode`."""
+    from openra_bench.rust_adapter import RustObsAdapter
+    from openra_rl_training.training.rust_env_pool import RustEnvPool
+
+    c = compile_level(load_pack(PACKS / _PACK), "medium")
+    c.fog_mode = mode
+    pool = RustEnvPool(size=1, scenario_path=_scenario_to_tmp_yaml(c))
+    ad = RustObsAdapter()
+    ad.observe(pool.acquire().reset(seed=1))
+    return ad.render_state()
+
+
+def test_perception_labels_are_friendly_and_unique():
+    from openra_bench.prompt_v2 import perception_labels
+
+    labels = perception_labels(_render_state("image-clear"))
+    assert labels, "expected labels for own + enemy actors"
+    # Each engine id maps to a distinct legible handle.
+    assert len(set(labels.values())) == len(labels)
+    assert any(v.startswith("tank-") for v in labels.values())
+    assert any(v.startswith("enemy-") for v in labels.values())
+
+
+def test_image_primary_briefing_has_no_coordinates():
+    """The redacted briefing must carry WHAT, never WHERE."""
+    import re
+
+    from openra_bench.prompt_v2 import (briefing_image_primary,
+                                        perception_labels)
+
+    rs = _render_state("image-clear")
+    text = briefing_image_primary(rs, perception_labels(rs))
+    # No `@(x,y)` coordinate tokens, and the enemy line is not enumerated.
+    assert not re.search(r"@\(\d+,\s*\d+\)", text), text
+    assert "8xe3" not in text and "1xbarr" not in text
+    assert "minimap" in text.lower()
+
+
+def test_image_primary_minimap_labels_every_unit():
+    from openra_bench.minimap import render_tactical_minimap
+    from openra_bench.prompt_v2 import perception_labels
+
+    rs = _render_state("image-clear")
+    labels = perception_labels(rs)
+    img = render_tactical_minimap(rs, scale=3, unit_labels=labels)
+    assert img is not None and img.size[0] > 0
+
+
+def test_to_commands_translates_labels_to_engine_ids():
+    """In the image channel the model passes minimap handles; the agent
+    maps them back to engine ids. Numeric ids pass straight through."""
+    import openra_train
+
+    from openra_bench.agent import _to_commands
+
+    Command = openra_train.Command
+    label_to_id = {"tank-1": "1004", "enemy-1": "1006"}
+    cmds = _to_commands(
+        [{"name": "attack_unit",
+          "arguments": {"unit_ids": ["tank-1"], "target_id": "enemy-1"}}],
+        Command, label_to_id,
+    )
+    assert '"1004"' in repr(cmds[0]) and '"1006"' in repr(cmds[0])
+    # Empty map (every other channel) — numeric ids untouched.
+    passthru = _to_commands(
+        [{"name": "stop", "arguments": {"unit_ids": [1004]}}], Command, {},
+    )
+    assert '"1004"' in repr(passthru[0])
+
+
+def test_image_primary_tools_retype_handles_as_strings():
+    from openra_bench.agent import _image_primary_tools, _tool_schemas
+
+    base = _tool_schemas(["move_units", "attack_unit"])
+    img = _image_primary_tools(base)
+    for t in img:
+        props = t["function"]["parameters"]["properties"]
+        if "unit_ids" in props:
+            assert props["unit_ids"]["items"]["type"] == "string"
+        if "target_id" in props:
+            assert props["target_id"]["type"] == "string"
+    # The base schemas are deep-copied, not mutated.
+    assert (
+        base[0]["function"]["parameters"]["properties"]
+        ["unit_ids"]["items"]["type"] == "integer"
+    )