Add inference code and MineExplorer patches

README.md

@@ -102,3 +102,68 @@ Perception prompts (thinking off, temperature 0, 320x180 frame):
 ## Prompt (B)
 
 `Task: find {target}. Reply with the behavior code for the next second.` The curiosity variant appends ` Seen directions (0, 45, ..., 315 deg right of view): {8 bits}. Visited cells 4 blocks along them: {8 bits}.`
+
+## 推理代码 / Inference code
+
+`code/` 是评测时实际用的脚本，原样上传，保留原来的相对目录（脚本之间按相对路径 import）。`patches/` 是对 MineExplorer 的两个补丁。
+
+- `code/train/ctrl_policy.py`：A 和 B 的策略。`Policy` 是 A（好奇心版按 `w0` 的行数自动加记忆输入；`box_joint_lora` 也用它），`BTPolicy` 是 B，`make_policy` 按 `kind` 选择。
+- `code/train/hid_policy.py`、`code/train/hid_server.py`：`hidden_frozen`、`hidden_joint_lora` 的策略和隐状态特征服务（HTTP，`--adapter` 加载 LoRA）。`hid_server.py` 的模型加载函数来自 `code/train/bt_train.py`。
+- `code/bench/diag_perception*.py`：感知问题原文、框的解析和视场常量，`ctrl_policy.py` 从这里 import；`code/train/grpo_locations.json` 在 import 时读取。
+- `code/bench/splits_find25.json`：find25 的 25 个任务编号。
+- `patches/MineExplorer.patch`：MineExplorer（commit `cca4c5e`）的 `eval_benchmark.py`，加了 `MC_POLICY` 入口（用上面的策略代替 LLM agent）、`--tasks`、多个 sandbox 轮流分配（`MC_SANDBOX_URLS`）；`env/minerl_sandbox.py` 的 reset 超时可配（`MC_RESET_TIMEOUT`）。
+- `patches/sandbox_mc_server.patch`：sandbox 镜像里的 `minestudio/mc_server.py`，step 返回 info（策略要读 `player_pos`），每次 reset 重新设 world seed。
+
+`code/` holds the scripts the evaluations ran, uploaded unchanged in their original relative layout (they import each other by relative path); `patches/` holds two patches to MineExplorer.
+
+- `code/train/ctrl_policy.py`: the policies of A and B. `Policy` is A (the curiosity variant turns on the memory inputs from the row count of `w0`; `box_joint_lora` uses it too), `BTPolicy` is B, and `make_policy` picks one by `kind`.
+- `code/train/hid_policy.py`, `code/train/hid_server.py`: the policy of `hidden_frozen` / `hidden_joint_lora` and the HTTP hidden-state server (`--adapter` loads a LoRA). `hid_server.py` takes its model loader from `code/train/bt_train.py`.
+- `code/bench/diag_perception*.py`: the perception prompts, the box parser and the field-of-view constants, imported by `ctrl_policy.py`; `code/train/grpo_locations.json` is read at import time.
+- `code/bench/splits_find25.json`: the 25 task ids of find25.
+- `patches/MineExplorer.patch`: MineExplorer (commit `cca4c5e`) `eval_benchmark.py` gains the `MC_POLICY` hook (runs the policies above instead of the LLM agent), `--tasks` and round-robin over several sandboxes (`MC_SANDBOX_URLS`); `env/minerl_sandbox.py` gets a configurable reset timeout (`MC_RESET_TIMEOUT`).
+- `patches/sandbox_mc_server.patch`: `minestudio/mc_server.py` in the sandbox image returns the step info (the policies read `player_pos`) and re-applies the world seed on every reset.
+
+Setup:
+
+```bash
+cd code
+git clone https://github.com/meituan-longcat/MineExplorer.git
+git -C MineExplorer checkout cca4c5e && git -C MineExplorer apply ../../patches/MineExplorer.patch
+mkdir -p models && ln -s ../../Qwen3.5-9B models/Qwen3.5-9B
+```
+
+Start the sandbox from MineExplorer's Docker image (`davidzhth/mineexplorer:0.0.1`, see its README) with `patches/sandbox_mc_server.patch` applied to `/home/sandbox/mc/lumine-client/minestudio/mc_server.py`. Versions used: vLLM 0.27.1; torch 2.13, transformers 5.15, peft 0.21 for `hid_server.py`; Python 3.11 with MineExplorer's requirements plus numpy, openai, requests for the eval.
+
+Perception server:
+
+```bash
+# A: base Qwen3.5-9B
+vllm serve models/Qwen3.5-9B --served-model-name perception --dtype bfloat16 --max-model-len 16384 \
+  --limit-mm-per-prompt '{"image": 24}' --port 9060
+# B or box_joint_lora: the LoRA under the name the policy asks for (box: ../ablations/box_joint_lora/grpo_r25)
+vllm serve models/Qwen3.5-9B --served-model-name perception-base --dtype bfloat16 --max-model-len 16384 \
+  --limit-mm-per-prompt '{"image": 24}' --port 9060 --enable-lora --max-lora-rank 16 \
+  --lora-modules perception=../B_behavior_token/grpo_r25
+# hidden_frozen (no adapter) and hidden_joint_lora
+CUDA_VISIBLE_DEVICES=0 python train/hid_server.py --port 9210 --adapter ../ablations/hidden_joint_lora/grpo_r25
+```
+
+Eval (find25, 1000 steps; our tables pool seeds 0, 10000, 20000; keep `--num-workers` at 2 or more, since 1 takes a serial path that ignores `MC_SANDBOX_URLS`):
+
+```bash
+cd MineExplorer
+MC_SANDBOX_URLS=http://localhost:8031,http://localhost:8032 MC_RESET_TIMEOUT=1200 \
+AGENT_API_KEY=unused AGENT_API_BASE=unused MC_POLICY='<json below>' \
+python eval_benchmark.py --model ctrl --benchmark-dir benchmark --output-dir out_s0 \
+  --tasks $(python -c "import json; print(','.join(json.load(open('../bench/splits_find25.json'))['eval']))") \
+  --num-workers 13 --max-steps 1000
+```
+
+| Model | Server | `MC_POLICY` |
+|---|---|---|
+| A (any `A_cerebellum/*/ctrl.npz`) | vLLM, base | `{"npz": "/abs/ctrl.npz", "perception": "vlm", "url": "http://localhost:9060/v1", "model": "perception", "seed": 0}` |
+| `box_joint_lora` | vLLM + its LoRA | as A, with its `ctrl.npz` |
+| B | vLLM + B's LoRA | `{"kind": "bt", "table": "/abs/table.npz", "url": "http://localhost:9060/v1", "model": "perception", "seed": 0}`, plus `"memory": true` for `grpo_r25_curiosity` |
+| `hidden_frozen`, `hidden_joint_lora` | `hid_server.py` (adapter only for `hidden_joint_lora`) | `{"kind": "hid", "npz": "/abs/ctrl.npz", "url": "http://localhost:9210", "seed": 0}` |
+
+Each episode writes `steps.jsonl` with the policy's per-step log (inputs, sampled bins, perception replies).

code/bench/diag_perception.py

@@ -0,0 +1,176 @@
+"""Diagnosis B (2026-10-13): can the base Qwen3.5-9B (no LoRA) recognize the target in our 320x180 frames?
+
+Frames: GRPO v2 rollouts, rounds 1-5 (round 6 was cut off mid-rollout when training was paused), frames/NNNN.png =
+the exact PNG bytes the policy saw. frames/i.png shows the pose after step i (steps.jsonl `pos`); frame 0 the
+teleport location (train/grpo_locations.json) with yaw 0, pitch 0, which is also the origin of the spawn-relative
+milestone targets. Angles follow benchmark_gen/utils.py (target yaw = atan2(-dx, dz)); a camera yaw delta adds to
+pos.yaw (checked on round 1: 6346 of 6346 turns), so a target at a larger yaw is to the right. Pitch > 0 looks down.
+- positives: target 3-12 blocks away, <= 20 deg left/right of the view center, <= 30 deg above/below it (the
+  vertical half field of view is 35: Minecraft fov 70, options.txt fov:0.0);
+- behind negatives: same distance band, >= 120 deg off center;
+- wrong-name negatives: the positive frames, asked about a target name of another scene that is neither a target of
+  this scene nor a block its build commands place.
+At most 2 frames per episode and target, >= 20 frames apart, then round-robin over scenes.
+Question "Is there a <name> in this image? Answer yes or no.", thinking off, temperature 0.
+Writes items.jsonl, summary.md and contact_sheet.png (48 positives, red circle = projected target point).
+Usage: diag_perception.py --url http://localhost:9041/v1 --model NAME [--n 200] [--out DIR]
+"""
+import argparse
+import base64
+from concurrent.futures import ThreadPoolExecutor
+import json
+import math
+from pathlib import Path
+import random
+
+from openai import OpenAI
+from PIL import Image, ImageDraw
+
+ROOT = Path(__file__).resolve().parents[1]
+GRPO = ROOT / "runs/grpo_v2_20261013"
+ROUNDS = [f"round{r:03d}" for r in range(1, 6)]
+BENCH = ROOT / "MineExplorer/benchmark"
+LOCATIONS = json.loads((ROOT / "train/grpo_locations.json").read_text())
+EYE = 1.62  # eye height above player_pos (the feet)
+W, H, HALF_VFOV = 320, 180, 35.0
+
+
+def episodes():
+    """Finished rollout episodes: (episode dir, scene, steps, poses, targets); poses[i] is the pose of frames/i.png,
+    targets are (milestone id, task, absolute target)."""
+    for result in sorted(p for r in ROUNDS for p in (GRPO / r).glob("*/result.json")):
+        ep = result.parent
+        scene = ep.name.split("_")[0]
+        x, y, z = LOCATIONS[scene]
+        steps = [json.loads(line) for line in (ep / "steps.jsonl").read_text().splitlines()]
+        poses = [{"x": x, "y": y, "z": z, "yaw": 0.0, "pitch": 0.0}] + [s["pos"] for s in steps]
+        meta = json.loads((BENCH / scene / "multi-agent/metadata.json").read_text())
+        targets = [(m["milestone_id"], m["task"], [t + o for t, o in zip(m["rules"][0]["params"]["target"], (x, y, z))])
+                   for m in meta["milestones"]["milestones"]]
+        yield ep, scene, steps, poses, targets
+
+
+def view(pose, target):
+    """(distance, horizontal, vertical): distance from the feet as the milestone check; horizontal = target yaw minus
+    view yaw in [-180, 180), > 0 = right; vertical = target elevation from the eye above the view center."""
+    dx, dy, dz = target[0] - pose["x"], target[1] - pose["y"] - EYE, target[2] - pose["z"]
+    h = (math.degrees(math.atan2(-dx, dz)) - pose["yaw"] + 180) % 360 - 180
+    v = math.degrees(math.atan2(dy, math.hypot(dx, dz))) + pose["pitch"]
+    return math.dist((pose["x"], pose["y"], pose["z"]), target), h, v
+
+
+def spread(candidates, n, rng):
+    """At most 2 per (episode, target), >= 20 frames apart, then round-robin over scenes up to n."""
+    rng.shuffle(candidates)
+    groups = {}
+    for c in candidates:
+        g = groups.setdefault((c["ep"], c["mid"]), [])
+        if len(g) < 2 and all(abs(c["frame"] - o["frame"]) >= 20 for o in g):
+            g.append(c)
+    by_scene = {}
+    for c in (c for g in groups.values() for c in g):
+        by_scene.setdefault(c["scene"], []).append(c)
+    queues = [by_scene[s] for s in sorted(by_scene)]
+    out = []
+    while len(out) < n and any(queues):
+        out += [q.pop() for q in queues if q][:n - len(out)]
+    return out
+
+
+def select(n):
+    """(positives, behind negatives, wrong-name negatives), n of each where available."""
+    rng = random.Random(0)
+    pos, behind, names, placed = [], [], {}, {}
+    for ep, scene, _, poses, targets in episodes():
+        placed[scene] = " ".join(json.loads((BENCH / scene / "multi-agent/metadata.json").read_text())["commands"])
+        for mid, task, target in targets:
+            if not task.startswith("find "):  # a handful of "interact with X" / "open door" goals
+                continue
+            name = task[5:].removeprefix("a ").replace("_", " ")
+            names.setdefault(scene, set()).add(name)
+            for i, pose in enumerate(poses):
+                dist, h, v = view(pose, target)
+                item = {"ep": str(ep.relative_to(ROOT)), "scene": scene, "mid": mid, "frame": i, "name": name,
+                        "asked": name, "dist": round(dist, 1), "h": round(h, 1), "v": round(v, 1)}
+                if 3 <= dist <= 12 and abs(h) <= 20 and abs(v) <= 30:
+                    pos.append(item)
+                elif 3 <= dist <= 12 and abs(h) >= 120:
+                    behind.append(item)
+    pos, behind = spread(pos, n, rng), spread(behind, n, rng)
+    every = sorted(set().union(*names.values()))
+    wrong = [{**p, "asked": rng.choice([x for x in every if x not in names[p["scene"]]
+                                        and x.replace(" ", "_") not in placed[p["scene"]]])} for p in pos]
+    return pos, behind, wrong
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--url", required=True)
+    ap.add_argument("--model", required=True)
+    ap.add_argument("--n", type=int, default=200)
+    ap.add_argument("--out", type=Path, default=ROOT / "runs/diag_20261013/perception")
+    args = ap.parse_args()
+    args.out.mkdir(parents=True, exist_ok=True)
+    pos, behind, wrong = select(args.n)
+    print(f"positives {len(pos)} from {len({p['scene'] for p in pos})} scenes, behind {len(behind)} from "
+          f"{len({p['scene'] for p in behind})} scenes, wrong-name {len(wrong)}", flush=True)
+
+    client = OpenAI(api_key="EMPTY", base_url=args.url)
+
+    def ask(kind_item):
+        kind, item = kind_item
+        png = (ROOT / item["ep"] / "frames" / f"{item['frame']:04d}.png").read_bytes()
+        r = client.chat.completions.create(
+            model=args.model, temperature=0, max_tokens=5,
+            messages=[{"role": "user", "content": [
+                {"type": "image_url", "image_url": {"url": "data:image/png;base64," + base64.b64encode(png).decode()}},
+                {"type": "text", "text": f"Is there a {item['asked']} in this image? Answer yes or no."}]}],
+            extra_body={"chat_template_kwargs": {"enable_thinking": False}})
+        answer = r.choices[0].message.content
+        return {"kind": kind, **item, "answer": answer, "yes": answer.strip().lower().startswith("yes")}
+
+    with ThreadPoolExecutor(16) as pool:
+        items = list(pool.map(ask, [("pos", p) for p in pos] + [("behind", b) for b in behind]
+                              + [("wrong", w) for w in wrong]))
+    with (args.out / "items.jsonl").open("w") as f:
+        f.writelines(json.dumps(item, ensure_ascii=False) + "\n" for item in items)
+
+    rate = lambda xs: f"{100 * sum(x['yes'] for x in xs) / len(xs):.1f}%" if xs else "-"
+    by_kind = {k: [x for x in items if x["kind"] == k] for k in ("pos", "behind", "wrong")}
+    lines = ["| 类型 | 数量 | 答 yes 的比例 |", "|---|---:|---:|",
+             f"| 正例（目标在视野中央附近） | {len(by_kind['pos'])} | {rate(by_kind['pos'])} |",
+             f"| 背后负例（目标在身后） | {len(by_kind['behind'])} | {rate(by_kind['behind'])} |",
+             f"| 错名负例（同一批正例帧，问别的场景的目标名） | {len(by_kind['wrong'])} | {rate(by_kind['wrong'])} |"]
+    per = sorted({x["asked"] for x in items}, key=lambda n: -sum(x["asked"] == n for x in by_kind["pos"]))[:20]
+    lines += ["", "逐目标（正例最多的 20 个；错名一列是把这个名字问在不含它的帧上）：", "",
+              "| 目标 | 正例 n | 正例 yes | 背后 n | 背后 yes | 错名 n | 错名 yes |", "|---|---:|---:|---:|---:|---:|---:|"]
+    for n in per:
+        cells = [[x for x in by_kind[k] if x["asked"] == n] for k in ("pos", "behind", "wrong")]
+        lines.append(f"| {n} | " + " | ".join(f"{len(c)} | {rate(c)}" for c in cells) + " |")
+    (args.out / "summary.md").write_text(
+        "帧来自 GRPO v2 第 1 到 5 轮 rollout（策略实际看到的 320×180 PNG）。正例：目标距离 3 到 12 格，偏离视野中心左右 ≤20°、"
+        "上下 ≤30°（竖直半视场 35°）；背后负例：同距离段，偏离 ≥120°；错名负例：同一批正例帧，问另一个场景的目标名"
+        "（本场景的目标和建造命令里都没有这种方块，但个别名字如南瓜、甘蔗可能在自然地形里出现）。每个回合每个目标最多 2 帧、"
+        f"相隔 ≥20 帧，按场景轮流抽：正例来自 {len({p['scene'] for p in pos})} 个场景，背后负例来自 "
+        f"{len({p['scene'] for p in behind})} 个场景。只按几何选帧，不判断遮挡；"
+        "接触表 contact_sheet.png 是前 48 个正例（红圈为目标点按视角投影的位置，标注为目标名、回答、距离、回合/帧号），"
+        "全部逐条结果见 items.jsonl。\n\n" + "\n".join(lines) + "\n")
+
+    sheet = Image.new("RGB", (6 * W, 8 * (H + 14)), "white")
+    for k, x in enumerate(by_kind["pos"][:48]):
+        img = Image.open(ROOT / x["ep"] / "frames" / f"{x['frame']:04d}.png").convert("RGB")
+        draw = ImageDraw.Draw(img)
+        # approximate pinhole projection from the two angles (exact at pitch 0); 16:9, vertical fov 70
+        u = W / 2 + W / 2 * math.tan(math.radians(x["h"])) / (math.tan(math.radians(HALF_VFOV)) * W / H)
+        v = H / 2 - H / 2 * math.tan(math.radians(x["v"])) / math.tan(math.radians(HALF_VFOV))
+        draw.ellipse((u - 6, v - 6, u + 6, v + 6), outline="red", width=2)
+        cell = (k % 6 * W, k // 6 * (H + 14))
+        sheet.paste(img, (cell[0], cell[1] + 14))
+        ImageDraw.Draw(sheet).text((cell[0] + 2, cell[1] + 1), f"{x['name']} | {x['answer'].strip()[:5]} | {x['dist']:.0f}m | "
+                                   f"{Path(x['ep']).name}/{x['frame']}", fill="black")
+    sheet.save(args.out / "contact_sheet.png")
+    print((args.out / "summary.md").read_text(), flush=True)
+
+
+if __name__ == "__main__":
+    main()

code/bench/diag_perception_direction.py

@@ -0,0 +1,196 @@
+"""Diagnosis B2 (2026-10-13): can the base Qwen3.5-9B (no LoRA) tell a controller WHERE the named target is in our
+320x180 frames, not only whether it is there?
+
+Frames, poses, angles and spreading come from bench/diag_perception.py (B), with wider bands:
+- positives: target 3-16 blocks away, |h| <= 45, |v| <= 30, 100 from each h bin [-45,-15), [-15,15], (15,45];
+- behind negatives: same distance band, |h| >= 120, 100;
+- wrong-name negatives: every third positive frame, asked about a target name of another scene that is neither a target
+  of this scene nor a block its build commands place (B's rule).
+Two questions per frame, thinking off, temperature 0:
+(a) grounding in Qwen's native JSON format. Qwen3.5 answers [{"bbox_2d": [x1, y1, x2, y2], "label": ...}] in relative
+    0-1000 coordinates (checked on six clearly visible frames of B: values up to 959 and 825 on a 320x180 frame that vLLM
+    resizes to 352x192; scaled by W/1000 and H/1000 the boxes sit on the objects). The first box counts; its center gives
+    a horizontal angle with B's pinhole model (exact at pitch 0);
+(b) one word: left, center, right or none (truth for positives: center = |h| <= 15).
+Writes items.jsonl, summary.md and contact_sheet.png (48 positives, 16 per h bin: green = boxes, the first one thick,
+red = projected target point as in B).
+Usage: diag_perception_direction.py --url http://localhost:9045/v1 --model NAME [--out DIR]
+"""
+import argparse
+import base64
+from concurrent.futures import ThreadPoolExecutor
+import json
+import math
+from pathlib import Path
+import random
+import re
+import statistics
+
+from openai import OpenAI
+from PIL import Image, ImageDraw
+
+from diag_perception import BENCH, H, HALF_VFOV, ROOT, W, episodes, spread, view
+
+TAN_HALF_HFOV = math.tan(math.radians(HALF_VFOV)) * W / H  # horizontal half fov ~51.2 deg
+SIDES = ("left", "center", "right")
+GROUND = ("Locate the {0} in this image and report its bbox coordinates in JSON format. "
+          "If there is no {0} in the image, output an empty list [].")
+WORD = ("Where is the {0} in this image? Answer with one word: left, center, or right. "
+        "Answer none if there is no {0} in the image.")
+BOX = re.compile(r'"bbox_2d"\s*:\s*\[\s*(-?[\d.]+)\s*,\s*(-?[\d.]+)\s*,\s*(-?[\d.]+)\s*,\s*(-?[\d.]+)\s*\]')
+
+
+def side(h):
+    return "left" if h < -15 else "center" if h <= 15 else "right"
+
+
+def select():
+    """(positives, behind negatives, wrong-name negatives)."""
+    rng = random.Random(0)
+    by_side, behind, names, placed = {s: [] for s in SIDES}, [], {}, {}
+    for ep, scene, _, poses, targets in episodes():
+        placed[scene] = " ".join(json.loads((BENCH / scene / "multi-agent/metadata.json").read_text())["commands"])
+        for mid, task, target in targets:
+            if not task.startswith("find "):
+                continue
+            name = task[5:].removeprefix("a ").replace("_", " ")
+            names.setdefault(scene, set()).add(name)
+            for i, pose in enumerate(poses):
+                dist, h, v = (round(a, 1) for a in view(pose, target))
+                item = {"ep": str(ep.relative_to(ROOT)), "scene": scene, "mid": mid, "frame": i, "name": name,
+                        "asked": name, "dist": dist, "h": h, "v": v, "pitch": round(pose["pitch"], 1)}
+                if 3 <= dist <= 16 and abs(h) <= 45 and abs(v) <= 30:
+                    by_side[side(h)].append(item)
+                elif 3 <= dist <= 16 and abs(h) >= 120:
+                    behind.append(item)
+    pos = [x for s in SIDES for x in spread(by_side[s], 100, rng)]
+    behind = spread(behind, 100, rng)
+    every = sorted(set().union(*names.values()))
+    wrong = [{**p, "asked": rng.choice([x for x in every if x not in names[p["scene"]]
+                                        and x.replace(" ", "_") not in placed[p["scene"]]])} for p in pos[::3]]
+    return pos, behind, wrong
+
+
+def spearman(a, b):
+    rank = lambda xs: [sorted(xs).index(x) + (sorted(xs).count(x) - 1) / 2 for x in xs]
+    return statistics.correlation(rank(a), rank(b))
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--url", required=True)
+    ap.add_argument("--model", required=True)
+    ap.add_argument("--out", type=Path, default=ROOT / "runs/diag_20261013/perception_direction")
+    args = ap.parse_args()
+    args.out.mkdir(parents=True, exist_ok=True)
+    pos, behind, wrong = select()
+    counts = ", ".join(f"{s} {sum(side(p['h']) == s for p in pos)}" for s in SIDES)
+    print(f"positives {len(pos)} ({counts}) from "
+          f"{len({p['scene'] for p in pos})} scenes, behind {len(behind)} from {len({b['scene'] for b in behind})} "
+          f"scenes, wrong-name {len(wrong)}", flush=True)
+
+    client = OpenAI(api_key="EMPTY", base_url=args.url)
+
+    def ask(kind_item):
+        kind, item = kind_item
+        png = (ROOT / item["ep"] / "frames" / f"{item['frame']:04d}.png").read_bytes()
+        image = {"type": "image_url", "image_url": {"url": "data:image/png;base64," + base64.b64encode(png).decode()}}
+
+        def chat(prompt, max_tokens):
+            r = client.chat.completions.create(
+                model=args.model, temperature=0, max_tokens=max_tokens,
+                messages=[{"role": "user", "content": [image, {"type": "text", "text": prompt.format(item["asked"])}]}],
+                extra_body={"chat_template_kwargs": {"enable_thinking": False}})
+            return r.choices[0].message.content
+
+        ground, word = chat(GROUND, 256), chat(WORD, 5)
+        boxes = [[float(c) for c in b] for b in BOX.findall(ground)]
+        out = {"kind": kind, **item, "ground": ground, "boxes": boxes, "word_raw": word,
+               "word": next((w for w in SIDES + ("none",) if word.strip().lower().startswith(w)), "other")}
+        if boxes:
+            x1, _, x2, _ = boxes[0]
+            out["h_box"] = round(math.degrees(math.atan(((x1 + x2) / 1000 - 1) * TAN_HALF_HFOV)), 1)
+        return out
+
+    with ThreadPoolExecutor(32) as pool:
+        items = list(pool.map(ask, [("pos", p) for p in pos] + [("behind", b) for b in behind]
+                              + [("wrong", w) for w in wrong]))
+    with (args.out / "items.jsonl").open("w") as f:
+        f.writelines(json.dumps(item, ensure_ascii=False) + "\n" for item in items)
+
+    by_kind = {k: [x for x in items if x["kind"] == k] for k in ("pos", "behind", "wrong")}
+    pct = lambda k, n: f"{100 * k / n:.1f}%" if n else "-"
+    med = lambda xs: f"{statistics.median(xs):+.1f}" if xs else "-"
+
+    def row(label, xs):
+        det = [x for x in xs if x["boxes"]]
+        err = [x["h_box"] - x["h"] for x in det]
+        said = [x for x in xs if x["word"] in SIDES]
+        return (f"| {label} | {len(xs)} | {pct(len(det), len(xs))} | {med([abs(e) for e in err]).lstrip('+')} | "
+                f"{med(err)} | {pct(sum(abs(e) <= 10 for e in err), len(det))} | "
+                f"{pct(sum(abs(e) <= 15 for e in err), len(det))} | "
+                f"{pct(sum(side(x['h_box']) == side(x['h']) for x in det), len(xs))} | {pct(len(said), len(xs))} | "
+                f"{pct(sum(x['word'] == side(x['h']) for x in said), len(xs))} | "
+                f"{pct(sum(x['word'] == side(x['h']) for x in said), len(said))} |")
+
+    p = by_kind["pos"]
+    lines = ["| 正例分组 | n | (a) 给框率 | (a) 角度误差中位数（度） | (a) 有符号误差中位数 | (a) 误差 ≤10° | (a) 误差 ≤15° | "
+             "(a) 框推出的方向对（占全部） | (b) 非 none 率 | (b) 方向对（占全部） | (b) 方向对（占作答） |",
+             "|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|", row("全部", p)]
+    lines += [row(f"距离 {lo}-{hi} 格", [x for x in p if lo <= x["dist"] < hi or (hi == 16 and x["dist"] == 16)])
+              for lo, hi in ((3, 6), (6, 10), (10, 16))]
+    lines += [row(f"h {label}", [x for x in p if side(x["h"]) == s])
+              for s, label in zip(SIDES, ("左 [-45,-15)", "中 [-15,15]", "右 (15,45]"))]
+    lines += ["", "(a) 的误差只算给了框的正例；“占全部”把没给框或答 none 算作错。", "",
+              "(b) 的混淆表（行为真值，列为回答）：", "", "| 真值 | left | center | right | none | 其他 |",
+              "|---|---:|---:|---:|---:|---:|"]
+    lines += [f"| {s} | " + " | ".join(str(sum(x["word"] == w for x in p if side(x["h"]) == s))
+                                       for w in SIDES + ("none", "other")) + " |" for s in SIDES]
+    lines += ["", "负例（误报率）：", "", "| 负例类型 | n | (a) 给框率 | (b) 非 none 率 |", "|---|---:|---:|---:|"]
+    lines += [f"| {label} | {len(by_kind[k])} | {pct(sum(bool(x['boxes']) for x in by_kind[k]), len(by_kind[k]))} | "
+              f"{pct(sum(x['word'] in SIDES for x in by_kind[k]), len(by_kind[k]))} |"
+              for k, label in (("behind", "背后（目标在身后）"), ("wrong", "错名（正例帧，问别的场景的目标名）"))]
+    det = [x for x in p if x["boxes"]]
+    height = [x["boxes"][0][3] - x["boxes"][0][1] for x in det]
+    area = [(x["boxes"][0][2] - x["boxes"][0][0]) * (x["boxes"][0][3] - x["boxes"][0][1]) for x in det]
+    dist = [x["dist"] for x in det]
+    lines += ["", f"距离线索（给了框的 {len(det)} 个正例，第一个框）：框高与几何距离的 Spearman ρ = "
+                  f"{spearman(height, dist):+.2f}，框面积与几何距离 ρ = {spearman(area, dist):+.2f}。"]
+    (args.out / "summary.md").write_text(
+        "帧和几何同 B（bench/diag_perception.py）：GRPO v2 第 1 到 5 轮 rollout 中策略实际看到的 320×180 PNG。"
+        "正例：目标距离 3 到 16 格，水平角 |h| ≤ 45°、竖直角 |v| ≤ 30°，按 h 分三段 [-45,-15)、[-15,15]、(15,45] 各抽 100；"
+        "背后负例：同距离段，|h| ≥ 120°，100 个；错名负例：每三个正例取一帧，问另一个场景的目标名（B 的规则），"
+        f"{len(by_kind['wrong'])} 个。每个回合每个目标在每段最多 2 帧、相隔 ≥20 帧，按场景轮流抽：正例来自 "
+        f"{len({x['scene'] for x in p})} 个场景、{len({x['ep'] for x in p})} 个回合，背后负例来自 "
+        f"{len({x['scene'] for x in by_kind['behind']})} 个场景。只按几何选帧，不判断遮挡。\n\n"
+        "坐标系核对：Qwen3.5 的原生定位输出是 JSON 列表 [{\"bbox_2d\": [x1, y1, x2, y2], \"label\": ...}]，坐标是相对 0 到 1000，"
+        "不是像素：在 B 的 6 张目标清楚可见的帧上，y 最大到 825、x 最大到 959（原图 320×180，vLLM 送进模型时缩放为 352×192，"
+        "两种像素坐标都不可能出现这些值），按 x·W/1000、y·H/1000 换算后框落在物体上（品红/紫色玻璃、箱子、淡蓝混凝土准确，"
+        "矢车菊和盆栽竹子这种很小的物体偏约 15 像素）。下面取第一个框的中心，用针孔模型（水平半视场 "
+        f"{math.degrees(math.atan(TAN_HALF_HFOV)):.1f}°，俯仰为 0 时精确）换算成水平角，和几何 h 比较。\n\n"
+        f"问题（思考关闭，温度 0）：(a) \"{GROUND.format('<name>')}\"；(b) \"{WORD.format('<name>')}\"，"
+        "真值 center 为 |h| ≤ 15°。\n\n" + "\n".join(lines) + "\n\n"
+        "接触表 contact_sheet.png：每段前 16 个正例（左、中、右依次排列），绿框为 (a) 的框（第一个加粗），红圈为目标点按视角投影的位置"
+        "（同 B），标注为序号、目标名、距离、h、(b) 的回答；全部逐条结果见 items.jsonl。\n")
+
+    sheet = Image.new("RGB", (6 * W, 8 * (H + 14)), "white")
+    shown = [x for s in SIDES for x in [x for x in p if side(x["h"]) == s][:16]]
+    for k, x in enumerate(shown):
+        img = Image.open(ROOT / x["ep"] / "frames" / f"{x['frame']:04d}.png").convert("RGB")
+        draw = ImageDraw.Draw(img)
+        u = W / 2 + W / 2 * math.tan(math.radians(x["h"])) / TAN_HALF_HFOV
+        v = H / 2 - H / 2 * math.tan(math.radians(x["v"])) / math.tan(math.radians(HALF_VFOV))
+        draw.ellipse((u - 6, v - 6, u + 6, v + 6), outline="red", width=2)
+        for j, (x1, y1, x2, y2) in enumerate(x["boxes"]):
+            draw.rectangle((x1 * W / 1000, y1 * H / 1000, x2 * W / 1000, y2 * H / 1000), outline=(0, 255, 0),
+                           width=2 if j == 0 else 1)
+        cell = (k % 6 * W, k // 6 * (H + 14))
+        sheet.paste(img, (cell[0], cell[1] + 14))
+        ImageDraw.Draw(sheet).text((cell[0] + 2, cell[1] + 1), f"{k} {x['name']} | {x['dist']:.0f}m | h{x['h']:+.0f} | "
+                                   f"{x['word_raw'].strip()[:7]}", fill="black")
+    sheet.save(args.out / "contact_sheet.png")
+    print((args.out / "summary.md").read_text(), flush=True)
+
+
+if __name__ == "__main__":
+    main()

code/bench/diag_perception_presence.py

@@ -0,0 +1,235 @@
+"""Verification step 1b (2026-10-13): presence scores of the base Qwen3.5-9B (no LoRA) from token logprobs, plus the
+box temporal check of PLAN_DECOUPLED_20261013.md 2.4 point 3. Thinking off, temperature 0, top 20 logprobs.
+
+1b, on step 1's 500 items (runs/diag_20261013/perception_direction/items.jsonl: 300 geometric positives, 100 behind,
+100 wrong-name), two scores:
+- s_yes: B's question, max_tokens 1; P(yes) / (P(yes) + P(no)), case and leading-space variants summed;
+- s_box: step 1's grounding question with max_tokens 48 (the controller's call). Answers either open a code fence
+  ("```", "json", "\\n", "[", "\\n", "\\t", '{"', ...) or refuse ("There are no objects belonging to ..."); after the fence
+  the list token is "[" with P("[]") ~ 0 (stored per item as p_list_empty), so empty vs non-empty is decided at the
+  first token: s_box = P(first token "```").
+AUC: 300 positives vs 200 negatives, and step 1's hand-labelled contact-sheet positives (37 visible vs 11 not).
+Temporal check on GRPO v2 rounds 1-5: disjoint windows of 5 consecutive frames turning >= 5 deg at each step.
+real = a target in the geometric field of view (|h| <= 51.2, |v| <= 35) 3-16 blocks away in all 5 frames;
+wrong = the same windows asked about a wrong name (step 1's rule); behind = a target 3-16 blocks away with |h| >= 120
+in all 5 frames (spread as step 1, up to 100). Per step, the first box centre's angle change is compared with
+-delta yaw, the change a fixed world point shows.
+Writes items.jsonl, windows.jsonl, summary.md.
+Usage: diag_perception_presence.py --url http://localhost:9041/v1 --model NAME [--out DIR]
+"""
+import argparse
+import base64
+from concurrent.futures import ThreadPoolExecutor
+import json
+import math
+from pathlib import Path
+import random
+import statistics
+
+from openai import OpenAI
+
+from diag_perception import BENCH, HALF_VFOV, ROOT, episodes, spread, view
+from diag_perception_direction import BOX, GROUND, SIDES, TAN_HALF_HFOV, side
+
+STEP1 = ROOT / "runs/diag_20261013/perception_direction/items.jsonl"
+HIDDEN = {11, 17, 19, 20, 27, 29, 33, 36, 43, 45, 47}  # contact-sheet positives judged not visible (step 1 summary.md)
+HALF_HFOV = math.degrees(math.atan(TAN_HALF_HFOV))
+YESNO = "Is there a {0} in this image? Answer yes or no."
+
+
+def windows(rng):
+    """(real, wrong, behind) windows: start frame, the 4 yaw changes, geometric (dist, h, v) per frame."""
+    real, behind, names, placed = [], [], {}, {}
+    for ep, scene, _, poses, targets in episodes():
+        placed[scene] = " ".join(json.loads((BENCH / scene / "multi-agent/metadata.json").read_text())["commands"])
+        turn = [(b["yaw"] - a["yaw"] + 180) % 360 - 180 for a, b in zip(poses, poses[1:])]
+        for mid, task, target in targets:
+            if not task.startswith("find "):
+                continue
+            name = task[5:].removeprefix("a ").replace("_", " ")
+            names.setdefault(scene, set()).add(name)
+            free = {"real": 0, "behind": 0}
+            for s in range(len(poses) - 4):
+                if not all(abs(d) >= 5 for d in turn[s:s + 4]):
+                    continue
+                geo = [[round(a, 1) for a in view(poses[s + j], target)] for j in range(5)]
+                kind = ("real" if all(3 <= d <= 16 and abs(h) <= HALF_HFOV and abs(v) <= HALF_VFOV for d, h, v in geo)
+                        else "behind" if all(3 <= d <= 16 and abs(h) >= 120 for d, h, v in geo) else None)
+                if kind and s >= free[kind]:
+                    free[kind] = s + 5
+                    (real if kind == "real" else behind).append(
+                        {"kind": kind, "ep": str(ep.relative_to(ROOT)), "scene": scene, "mid": mid, "frame": s,
+                         "name": name, "asked": name, "turn": [round(d, 2) for d in turn[s:s + 4]], "geo": geo})
+    every = sorted(set().union(*names.values()))
+    wrong = [{**w, "kind": "wrong", "asked": rng.choice([x for x in every if x not in names[w["scene"]]
+                                                         and x.replace(" ", "_") not in placed[w["scene"]]])}
+             for w in real]
+    return real, wrong, spread(behind, 100, rng)
+
+
+def auc(pos, neg):
+    return sum((a > b) + 0.5 * (a == b) for a in pos for b in neg) / (len(pos) * len(neg))
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--url", required=True)
+    ap.add_argument("--model", required=True)
+    ap.add_argument("--out", type=Path, default=ROOT / "runs/diag_20261013/perception_presence")
+    args = ap.parse_args()
+    args.out.mkdir(parents=True, exist_ok=True)
+    step1 = [json.loads(line) for line in STEP1.read_text().splitlines()]
+    real, wrong, behind = windows(random.Random(0))
+    print(f"windows: real {len(real)} ({len({w['scene'] for w in real})} scenes, {len({w['ep'] for w in real})} "
+          f"episodes), wrong {len(wrong)}, behind {len(behind)} ({len({w['scene'] for w in behind})} scenes)", flush=True)
+
+    client = OpenAI(api_key="EMPTY", base_url=args.url)
+
+    def chat(key, prompt, max_tokens):
+        ep, frame, asked = key
+        png = (ROOT / ep / "frames" / f"{frame:04d}.png").read_bytes()
+        r = client.chat.completions.create(
+            model=args.model, temperature=0, max_tokens=max_tokens, logprobs=True, top_logprobs=20,
+            messages=[{"role": "user", "content": [
+                {"type": "image_url", "image_url": {"url": "data:image/png;base64," + base64.b64encode(png).decode()}},
+                {"type": "text", "text": prompt.format(asked)}]}],
+            extra_body={"chat_template_kwargs": {"enable_thinking": False}})
+        return r.choices[0].message.content, r.choices[0].logprobs.content
+
+    def ground(key):
+        text, lp = chat(key, GROUND, 48)
+        boxes = [[float(c) for c in b] for b in BOX.findall(text)]
+        out = {"ground": text, "boxes": boxes,
+               "s_box": sum(math.exp(t.logprob) for t in lp[0].top_logprobs if t.token.strip() == "```"),
+               "p_list_empty": sum(math.exp(t.logprob) for t in lp[3].top_logprobs if t.token == "[]")
+               if text.startswith("```json\n[") else None}
+        if boxes:
+            x1, _, x2, _ = boxes[0]
+            out["h_box"] = round(math.degrees(math.atan(((x1 + x2) / 1000 - 1) * TAN_HALF_HFOV)), 1)
+        return out
+
+    def yes(key):
+        text, lp = chat(key, YESNO, 1)
+        p = {w: sum(math.exp(t.logprob) for t in lp[0].top_logprobs if t.token.strip().lower() == w)
+             for w in ("yes", "no")}
+        return {"answer": text, "s_yes": p["yes"] / (p["yes"] + p["no"])}
+
+    keys1 = [(x["ep"], x["frame"], x["asked"]) for x in step1]
+    keys = list(dict.fromkeys(keys1 + [(w["ep"], w["frame"] + j, w["asked"]) for w in real + wrong + behind
+                                       for j in range(5)]))
+    with ThreadPoolExecutor(32) as pool:
+        g = dict(zip(keys, pool.map(ground, keys)))
+        y = dict(zip(keys1, pool.map(yes, keys1)))
+    items = [{**{k: x[k] for k in ("kind", "ep", "scene", "mid", "frame", "name", "asked", "dist", "h", "v", "pitch")},
+              **y[key], **g[key]} for x, key in zip(step1, keys1)]
+    wins = [{**w, "ground": [g[(w["ep"], w["frame"] + j, w["asked"])] for j in range(5)]}
+            for w in real + wrong + behind]
+    real, wrong, behind = ([w for w in wins if w["kind"] == k] for k in ("real", "wrong", "behind"))
+    with (args.out / "items.jsonl").open("w") as f:
+        f.writelines(json.dumps(x, ensure_ascii=False) + "\n" for x in items)
+    with (args.out / "windows.jsonl").open("w") as f:
+        f.writelines(json.dumps(w, ensure_ascii=False) + "\n" for w in wins)
+
+    by = {k: [x for x in items if x["kind"] == k] for k in ("pos", "behind", "wrong")}
+    shown = [x for s in SIDES for x in [x for x in by["pos"] if side(x["h"]) == s][:16]]
+    vis = [x for k, x in enumerate(shown) if k not in HIDDEN]
+    hid = [x for k, x in enumerate(shown) if k in HIDDEN]
+    neg = by["behind"] + by["wrong"]
+    aucs = ["| 比较 | n | s_yes AUC | s_box AUC |", "|---|---|---:|---:|"]
+    aucs += [f"| {label} | {len(a)} 对 {len(b)} | {auc([x['s_yes'] for x in a], [x['s_yes'] for x in b]):.3f} | "
+             f"{auc([x['s_box'] for x in a], [x['s_box'] for x in b]):.3f} |"
+             for label, a, b in (("正例对负例（背后 + 错名）", by["pos"], neg), ("正例对背后", by["pos"], by["behind"]),
+                                 ("正例对错名", by["pos"], by["wrong"]), ("人工：可见对看不见", vis, hid),
+                                 ("人工可见对负例", vis, neg))]
+    med = lambda xs: f"{statistics.median(xs):.3f}"
+    scores = ["| 组 | n | s_yes 中位数 | 答 yes | s_box 中位数 | 给框（48 token） |", "|---|---:|---:|---:|---:|---:|"]
+    scores += [f"| {label} | {len(xs)} | {med([x['s_yes'] for x in xs])} | "
+               f"{sum(x['answer'].strip().lower().startswith('yes') for x in xs)} | {med([x['s_box'] for x in xs])} | "
+               f"{sum(bool(x['boxes']) for x in xs)} |"
+               for label, xs in (("正例", by["pos"]), ("背后", by["behind"]), ("错名", by["wrong"]),
+                                 ("人工可见", vis), ("人工看不见", hid))]
+    fence = [x["p_list_empty"] for x in items if x["p_list_empty"] is not None]
+    starts = sorted({x["ground"][:12] for x in items if x["p_list_empty"] is None})
+    scores += ["", f"代码块开头的回答 {len(fence)} 条，列表 token 上 P(\"[]\") 最大 {max(fence):.1e}；"
+                   f"其余 {len(items) - len(fence)} 条开头为 {starts}。"]
+
+    def row(label, seqs):
+        """seqs: (4 yaw changes, 5 box centre angles, None = no box)."""
+        had = [(-d, a, b) for turns, hs in seqs for d, a, b in zip(turns, hs, hs[1:]) if a is not None]
+        both = [(x, b - a) for x, a, b in had if b is not None]
+        slope, icpt = statistics.linear_regression(*zip(*both))
+        rms = math.sqrt(statistics.fmean((y - slope * x - icpt) ** 2 for x, y in both))
+        frames = [h for _, hs in seqs for h in hs]
+        big = [y for x, y in both if abs(x) > 10]
+        pct = lambda k, n: f"{100 * k / n:.0f}%"
+        return (f"| {label} | {len(seqs)} | {pct(sum(h is not None for h in frames), len(frames))} | "
+                f"{pct(sum(b is None for _, _, b in had), len(had))} | {len(both)} | "
+                f"{statistics.median(abs(x) for x, _ in both):.1f} | {slope:.2f} | {icpt:+.1f} | {rms:.1f} | "
+                f"{pct(sum(abs(y - x) <= 5 for x, y in both), len(both))} | {pct(sum(abs(y) <= 5 for y in big), len(big))} | "
+                f"{pct(sum(min(abs(y - x), abs(y)) > 10 for x, y in both), len(both))} | "
+                f"{statistics.median(abs(y) for _, y in both):.1f} |")
+
+    boxes = lambda ws: [(w["turn"], [f.get("h_box") for f in w["ground"]]) for w in ws]
+    rng = random.Random(0)
+    pool = [f["h_box"] for w in wrong + behind for f in w["ground"] if "h_box" in f]
+    errs = [x["h_box"] - x["h"] for x in by["pos"] if "h_box" in x]
+
+    def fake_sim(redraw, world):
+        """Fake box angles over the fake windows' turns (20 repeats): first from the observed fake boxes; each step
+        redrawn with probability redraw, else kept (world = 1: shifted by -delta yaw like a fixed world point)."""
+        seqs = []
+        for w in (wrong + behind) * 20:
+            hs = [rng.choice(pool)]
+            for d in w["turn"]:
+                hs.append(rng.choice(pool) if rng.random() < redraw else hs[-1] - world * d)
+            seqs.append((w["turn"], hs))
+        return seqs
+
+    def real_sim(redraw):
+        """True target angle + an error from the 300 positives (plan 2.4 point 1), redrawn with probability redraw."""
+        seqs = []
+        for w in real * 20:
+            e, hs = rng.choice(errs), []
+            for _, h, _ in w["geo"]:
+                e = rng.choice(errs) if hs and rng.random() < redraw else e
+                hs.append(h + e)
+            seqs.append((w["turn"], hs))
+        return seqs
+
+    temporal = ["| 窗口 | 窗口数 | 帧给框 | 消失（上一帧有框，这一帧没有） | 相邻两帧都有框的步 | \\|Δyaw\\| 中位数 | 斜率 | 截距 | "
+                "残差 RMS | 随世界点（\\|Δh_box + Δyaw\\| ≤ 5°） | 画面不动（\\|Δh_box\\| ≤ 5°，\\|Δyaw\\| > 10° 的步） | "
+                "跳（两者都偏 >10°） | \\|Δh_box\\| 中位数 |",
+                "|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|",
+                row("真目标：几何真值 h（参照）", [(w["turn"], [g[1] for g in w["geo"]]) for w in real]),
+                row("真目标：框", boxes(real)), row("假框：错名（同一批窗口）", boxes(wrong)),
+                row("假框：背后", boxes(behind)), row("假框合计", boxes(wrong + behind)),
+                row("模拟假框 (1) 每步独立", fake_sim(1, 0)), row("模拟假框 (2) 固定世界点", fake_sim(0, 1)),
+                row("模拟假框 (3) 画面里不动", fake_sim(0, 0)), row("模拟假框 (3)，每步以 0.5 概率重抽", fake_sim(0.5, 0)),
+                row("模拟真框：每步独立误差（2.4 第 1 点原样）", real_sim(1)), row("模拟真框：误差保持，每步以 0.5 概率重抽", real_sim(0.5))]
+    (args.out / "summary.md").write_text(
+        "基座 Qwen3.5-9B（无 LoRA），思考关闭，温度 0，取前 20 个 token 的 logprob。\n\n"
+        "## 1b 存在性分数\n\n"
+        "同第 1 步的 500 条（runs/diag_20261013/perception_direction/items.jsonl：几何正例 300，背后负例 100，错名负例 100）。"
+        f"s_yes：B 的问题 \"{YESNO.format('<name>')}\"，max_tokens 1，P(yes)/(P(yes)+P(no))，大小写和前导空格变体合计。"
+        "s_box：第 1 步的定位问题，max_tokens 48（2.2 里控制器的调用）。回答只有两种开头：代码块（```、json、换行、[、换行、制表符、{\"…）"
+        "或拒答（There are no objects belonging to …）；进了代码块后列表 token 是 [，P(\"[]\") 约为 0，"
+        "所以空和非空在第一个 token 上决定：s_box = P(第一个 token 是 ```)。人工标注取第 1 步 summary.md 接触表核对："
+        "48 个正例里序号 11、17、19、20、27、29、33、36、43、45、47 看不见，其余 37 个可见。\n\n"
+        + "\n".join(aucs) + "\n\n" + "\n".join(scores) + "\n\n"
+        "## 框的时间行为\n\n"
+        "GRPO v2 第 1 到 5 轮 rollout，5 帧连续窗口，4 步每步 |Δyaw| ≥ 5°，同一回合同一目标的窗口不重叠。"
+        f"真目标：目标在全部 5 帧都在几何视场内（|h| ≤ {HALF_HFOV:.1f}°，|v| ≤ 35°）、距离 3 到 16 格，全部取用 {len(real)} 个"
+        f"（{len({w['scene'] for w in real})} 个场景、{len({w['ep'] for w in real})} 个回合，这批 rollout 里满足条件的只有这么多）；"
+        "错名：同一批窗口问别的场景的目标名（第 1 步的规则）；背后：目标在全部 5 帧都在 3 到 16 格、|h| ≥ 120°，按第 1 步的 spread 取"
+        f" {len(behind)} 个（{len({w['scene'] for w in behind})} 个场景）。定位问题同 1b（max_tokens 48），取第一个框中心换算水平角 h_box。"
+        "每步把 Δh_box 对 −Δyaw（固定世界点在画面里的水平角变化）做最小二乘：固定世界点斜率 1，画面里不动斜率 0。"
+        "参照行用目标点的几何 h 代替框，差别来自走路的视差。跳：框中心变化离固定世界点的预测和画面不动都超过 10°。"
+        "模拟行在观测窗口的转头序列上各重复 20 次（seed 0）：假框的位置从观测到的假框 h_box 里抽，真框 = 目标几何 h + 从 1b 的 300 个正例里抽的"
+        "框误差；“重抽”指这一步换一个新抽样，否则沿用上一步（假框 (2) 再按 −Δyaw 平移）。\n\n"
+        + "\n".join(temporal) + "\n\n"
+        "逐条结果：items.jsonl（1b 的 500 条），windows.jsonl（每个窗口 5 帧的回答、框、h_box、s_box）。\n")
+    print((args.out / "summary.md").read_text(), flush=True)
+
+
+if __name__ == "__main__":
+    main()

code/bench/splits_find25.json

@@ -0,0 +1,30 @@
+{
+ "eval": [
+  "0046",
+  "0048",
+  "0050",
+  "0062",
+  "0066",
+  "0084",
+  "0116",
+  "0147",
+  "0148",
+  "0158",
+  "0165",
+  "0188",
+  "0198",
+  "0208",
+  "0233",
+  "0237",
+  "0243",
+  "0261",
+  "0310",
+  "0313",
+  "0329",
+  "0348",
+  "0356",
+  "0377",
+  "0386"
+ ],
+ "note": "First 25 tasks of splits_find.json eval in its fixed sorted order (hop mix 10/7/5/3 vs 20/15/10/5 in the 50). Fixed before any 25-task result; chosen 2026-10-11 so the running 20-frame eval (which dispatches in this order) can stop at 25."
+}

code/train/bt_train.py

@@ -0,0 +1,274 @@
+"""LoRA training of the BT policy (EXPERIMENT_LOG.md 2026-10-17): Qwen3.5-9B reads one frame and BT_PROMPT with the
+target name and writes the behavior code (one digit token per FSQ dimension). Runs with torchrun in .train-venv.
+
+Rows (jsonl): image (PNG path), name, code (digits); ppo rows also adv and vllm_logp, and memory (the two 0/1 strings
+of the decision's prompt) when the rollouts ran with memory. A row is the chat template of [image, bt_prompt(name,
+memory)] with the generation prompt (thinking off), as vLLM renders BTPolicy's request, followed by the code and
+<|im_end|>; batches are right-padded and the scored tokens gathered per row.
+- sft: cross entropy on the code tokens and <|im_end|>, new LoRA (r 16, alpha 32, train_iql.py's modules), AdamW,
+  10-update warmup then cosine; per-digit and whole-code (all digits argmax) accuracy on --valid after each epoch.
+  Saves the adapter to --out (vLLM-loadable).
+- ppo: continues --adapter (and its optimizer.pt if it has one, i.e. a PPO round's), loads --ref (the SFT adapter) as
+  a second adapter; old and ref log-probs of the code tokens computed once with no_grad, then --epochs of PPO:
+  loss = -min(ratio A, clip(ratio, 0.8, 1.2) A) + beta * sum over the code tokens of k3 = exp(ref - lp) - (ref - lp) - 1,
+  ratio = exp(sum lp - sum old). Saves the adapter, optimizer.pt and stats.json (pg_loss, kl, clip_frac of the last
+  epoch; logp_gap = mean of old - vllm_logp, logp_gap_abs) to --out.
+Usage: CUDA_VISIBLE_DEVICES=6,7 .train-venv/bin/python -m torch.distributed.run --nproc_per_node=2 train/bt_train.py
+         sft --data TRAIN.jsonl --valid VALID.jsonl --out DIR [--epochs 1 --lr 1e-4 --max-updates N]
+       ... train/bt_train.py ppo --data ROUND/train.jsonl --adapter ADAPTER --ref SFT --out DIR
+"""
+import argparse
+import json
+import math
+import os
+import random
+import time
+from pathlib import Path
+
+import torch
+import torch.distributed as dist
+from peft import LoraConfig, PeftModel, get_peft_model
+from PIL import Image
+from torch.nn.parallel import DistributedDataParallel
+from torch.utils.data import DataLoader, DistributedSampler
+from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration
+
+from ctrl_policy import bt_prompt
+
+ROOT = Path(__file__).resolve().parents[1]
+MODULES = (r"model.language_model.*\.(q_proj|k_proj|v_proj|o_proj|in_proj_qkv|in_proj_z|in_proj_b|in_proj_a|out_proj"
+           r"|gate_proj|up_proj|down_proj)$")  # train_iql.py
+END = "<|im_end|>"
+CLIP, BETA = 0.2, 0.02
+
+
+def log(message):
+    print(f"{time.strftime('%Y-%m-%d %H:%M:%S')} {message}", flush=True)
+
+
+class Collate:
+    def __init__(self, processor):
+        self.processor = processor
+
+    def __call__(self, rows):
+        p = self.processor
+        texts = [p.apply_chat_template([{"role": "user", "content": [{"type": "image"}, {"type": "text",
+                                         "text": bt_prompt(r["name"], r.get("memory"))}]}], tokenize=False,
+                                       add_generation_prompt=True, enable_thinking=False) + r["code"] + END
+                 for r in rows]
+        inputs = p(text=texts, images=[Image.open(r["image"]).convert("RGB") for r in rows], padding=True,
+                   padding_side="right", return_tensors="pt")
+        n = len(rows[0]["code"])
+        ends = inputs["attention_mask"].sum(1)  # sequence lengths: the last token, <|im_end|>, is at end - 1
+        targets = torch.stack([inputs["input_ids"][i, e - n - 1:e] for i, e in enumerate(ends)])  # code + im_end
+        extra = {k: torch.tensor([r[k] for r in rows], dtype=torch.float32) for k in ("adv", "vllm_logp", "index")
+                 if k in rows[0]}
+        return {"inputs": dict(inputs), "ends": ends, "targets": targets, **extra}
+
+
+def token_logprobs(model, batch, device):
+    """(B, n + 1) log-probs of the code tokens and <|im_end|>, and the logits' argmax at those positions."""
+    inputs = {k: v.to(device) for k, v in batch["inputs"].items()}
+    ends, k = batch["ends"], batch["targets"].shape[1]
+    lo, hi = int(ends.min()) - k - 1, int(ends.max()) - 1  # logits at position j predict token j + 1
+    with torch.autocast("cuda", dtype=torch.bfloat16):
+        logits = model(**inputs, logits_to_keep=torch.arange(lo, hi, device=device), use_cache=False).logits
+    rows = torch.stack([logits[i, e - k - 1 - lo:e - 1 - lo] for i, e in enumerate(ends.tolist())]).float()
+    lp = rows.log_softmax(-1)
+    targets = batch["targets"].to(device)
+    return lp.gather(-1, targets[..., None])[..., 0], rows.argmax(-1) == targets
+
+
+def load_model(local):
+    model = Qwen3_5ForConditionalGeneration.from_pretrained(
+        ROOT / "models/Qwen3.5-9B", dtype=torch.bfloat16, device_map={"": local}, attn_implementation="sdpa",
+        local_files_only=True, use_kernels=True)
+    model.requires_grad_(False)
+    return model
+
+
+def train_setup(model):
+    model.enable_input_require_grads()
+    model.gradient_checkpointing_enable(gradient_checkpointing_kwargs={"use_reentrant": False})
+    model.config.use_cache = False
+    model.config.text_config.use_cache = False
+
+
+def loader(rows, collate, batch, rank, world, seed, shuffle=True):
+    sampler = DistributedSampler(rows, num_replicas=world, rank=rank, seed=seed, shuffle=shuffle)
+    return sampler, DataLoader(rows, batch_size=batch, sampler=sampler, collate_fn=collate, num_workers=4,
+                               persistent_workers=True, prefetch_factor=2)
+
+
+@torch.no_grad()
+def accuracy(model, rows, collate, batch, rank, world, device):
+    """Per-digit and whole-code accuracy (teacher-forced argmax) of the code tokens on rows."""
+    model.eval()
+    totals = torch.zeros(3, device=device)
+    for b in DataLoader(rows[rank::world], batch_size=batch, collate_fn=collate, num_workers=4):
+        _, right = token_logprobs(model, b, device)
+        right = right[:, :-1]
+        totals += torch.tensor([right.sum(), right.all(1).sum(), len(right)], device=device, dtype=torch.float32)
+    dist.all_reduce(totals)
+    model.train()
+    return {"digit_acc": round((totals[0] / totals[2] / right.shape[1]).item(), 4),
+            "code_acc": round((totals[1] / totals[2]).item(), 4), "rows": int(totals[2])}
+
+
+def sft(args, rank, world, local, device):
+    rows = [json.loads(l) for l in args.data.read_text().splitlines()]
+    valid = [json.loads(l) for l in args.valid.read_text().splitlines()]
+    processor = AutoProcessor.from_pretrained(ROOT / "models/Qwen3.5-9B", local_files_only=True)
+    collate = Collate(processor)
+    torch.manual_seed(args.seed)
+    model = get_peft_model(load_model(local), LoraConfig(r=16, lora_alpha=32, lora_dropout=0.0,
+                                                         task_type="CAUSAL_LM", target_modules=MODULES))
+    train_setup(model)
+    ddp = DistributedDataParallel(model, device_ids=[local])
+    sampler, data = loader(rows, collate, args.batch, rank, world, args.seed)
+    opt = torch.optim.AdamW((p for p in model.parameters() if p.requires_grad), lr=args.lr)
+    total = min(args.max_updates or 10**9, len(data) * args.epochs)
+    if rank == 0:
+        log(f"SFT rows {len(rows)} valid {len(valid)}, {world} GPUs x batch {args.batch}, updates {total}")
+        log("VALID before " + json.dumps(accuracy(model, valid, collate, args.batch, rank, world, device)))
+    else:
+        accuracy(model, valid, collate, args.batch, rank, world, device)
+    update, start = 0, time.monotonic()
+    for epoch in range(args.epochs):
+        sampler.set_epoch(epoch)
+        for b in data:
+            lp, _ = token_logprobs(ddp, b, device)
+            loss = -lp.mean()
+            opt.zero_grad(set_to_none=True)
+            loss.backward()
+            torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
+            for g in opt.param_groups:
+                g["lr"] = args.lr * min(1, (update + 1) / 10) * 0.5 * (1 + math.cos(math.pi * update / total))
+            opt.step()
+            update += 1
+            if rank == 0 and (update <= 5 or update % 50 == 0):
+                s = time.monotonic() - start
+                log(f"SFT update {update}/{total} loss {loss.item():.4f} {s / update:.2f} s/update "
+                    f"eta {(total - update) * s / update / 60:.0f} min peak {torch.cuda.max_memory_allocated() / 2**30:.1f} GiB")
+            if update >= total:
+                break
+        acc = accuracy(model, valid, collate, args.batch, rank, world, device)
+        if rank == 0:
+            log(f"VALID epoch {epoch + 1} " + json.dumps(acc))
+            model.save_pretrained(args.out)
+            (args.out / "valid.json").write_text(json.dumps({"epoch": epoch + 1, "updates": update, **acc}))
+        if update >= total:
+            break
+
+
+def ppo(args, rank, world, local, device):
+    rows = [{**json.loads(l), "index": i} for i, l in enumerate(args.data.read_text().splitlines())]
+    processor = AutoProcessor.from_pretrained(ROOT / "models/Qwen3.5-9B", local_files_only=True)
+    collate = Collate(processor)
+    torch.manual_seed(args.seed)
+    model = PeftModel.from_pretrained(load_model(local), args.adapter, is_trainable=True)
+    model.load_adapter(args.ref, adapter_name="ref")
+    n = len(rows[0]["code"])
+    lp_of = {}
+    for name in ("ref", "default"):  # once per round, no_grad: the reference and the old policy
+        model.set_adapter(name)
+        model.eval()
+        mine = {}
+        with torch.no_grad():
+            for b in DataLoader(rows[rank::world], batch_size=2 * args.batch, collate_fn=collate, num_workers=4):
+                lp, _ = token_logprobs(model, b, device)
+                mine.update(zip(b["index"].long().tolist(), lp[:, :n].cpu()))
+        gathered = [None] * world
+        dist.all_gather_object(gathered, mine)
+        lp_of[name] = torch.stack([{k: v for g in gathered for k, v in g.items()}[i] for i in range(len(rows))])
+    model.set_adapter("default")
+    for name, p in model.named_parameters():
+        p.requires_grad = "lora_" in name and ".default." in name
+    model.train()
+    train_setup(model)
+    ddp = DistributedDataParallel(model, device_ids=[local])
+    params = [p for p in model.parameters() if p.requires_grad]
+    opt = torch.optim.AdamW(params, lr=args.lr)
+    if (args.adapter / "optimizer.pt").exists():
+        opt.load_state_dict(torch.load(args.adapter / "optimizer.pt", map_location=device))
+        for g in opt.param_groups:
+            g["lr"] = args.lr
+    sampler, data = loader(rows, collate, args.batch, rank, world, args.seed)
+    old_all, ref_all = lp_of["default"].to(device), lp_of["ref"].to(device)
+    vllm = torch.tensor([r["vllm_logp"] for r in rows], device=device)
+    gap = old_all.sum(1) - vllm
+    start = time.monotonic()
+    for epoch in range(args.epochs):
+        sampler.set_epoch(epoch)
+        stats = []
+        for step, b in enumerate(data):
+            idx = b["index"].long().to(device)
+            lp, _ = token_logprobs(ddp, b, device)
+            lp = lp[:, :n]
+            adv = b["adv"].to(device)
+            ratio = (lp.sum(1) - old_all[idx].sum(1)).exp()
+            pg = -torch.min(ratio * adv, ratio.clamp(1 - CLIP, 1 + CLIP) * adv)
+            d = ref_all[idx] - lp
+            kl = (d.exp() - d - 1).sum(1)
+            loss = (pg + BETA * kl).mean() / args.accumulation
+            sync = (step + 1) % args.accumulation == 0 or step + 1 == len(data)
+            if sync:
+                loss.backward()
+                torch.nn.utils.clip_grad_norm_(params, 1.0)
+                opt.step()
+                opt.zero_grad(set_to_none=True)
+            else:
+                with ddp.no_sync():
+                    loss.backward()
+            stats.append([pg.mean().item(), kl.mean().item(), ((ratio - 1).abs() > CLIP).float().mean().item()])
+        if rank == 0:
+            log(f"PPO epoch {epoch + 1} pg {sum(s[0] for s in stats) / len(stats):+.4f} "
+                f"kl {sum(s[1] for s in stats) / len(stats):.4f} {time.monotonic() - start:.0f} s")
+    totals = torch.tensor(stats, device=device).mean(0)
+    dist.all_reduce(totals)
+    totals /= world
+    if rank == 0:
+        args.out.mkdir(parents=True, exist_ok=True)
+        model.save_pretrained(args.out, selected_adapters=["default"])
+        torch.save(opt.state_dict(), args.out / "optimizer.pt")
+        result = {"pg_loss": round(totals[0].item(), 4), "kl": round(totals[1].item(), 4),
+                  "clip_frac": round(totals[2].item(), 4), "logp_gap": round(gap.mean().item(), 4),
+                  "logp_gap_abs": round(gap.abs().mean().item(), 4), "rows": len(rows),
+                  "train_s": round(time.monotonic() - start)}
+        (args.out / "stats.json").write_text(json.dumps(result))
+        log("PPO_DONE " + json.dumps(result))
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("mode", choices=["sft", "ppo"])
+    ap.add_argument("--data", type=Path, required=True)
+    ap.add_argument("--valid", type=Path)
+    ap.add_argument("--adapter", type=Path)
+    ap.add_argument("--ref", type=Path)
+    ap.add_argument("--out", type=Path, required=True)
+    ap.add_argument("--epochs", type=int)
+    ap.add_argument("--lr", type=float)
+    ap.add_argument("--batch", type=int, default=16, help="rows per GPU per forward")
+    ap.add_argument("--accumulation", type=int, default=2, help="ppo: forwards per update (64 rows on 2 GPUs)")
+    ap.add_argument("--max-updates", type=int)
+    ap.add_argument("--seed", type=int, default=0)
+    args = ap.parse_args()
+    rank, world, local = int(os.environ["RANK"]), int(os.environ["WORLD_SIZE"]), int(os.environ["LOCAL_RANK"])
+    torch.set_num_threads(4)
+    random.seed(args.seed)
+    torch.cuda.set_device(local)
+    device = torch.device("cuda", local)
+    dist.init_process_group("nccl", device_id=device)
+    if args.mode == "sft":
+        args.epochs, args.lr = args.epochs or 1, args.lr or 1e-4
+        sft(args, rank, world, local, device)
+    else:
+        args.epochs, args.lr = args.epochs or 2, args.lr or 3e-6
+        ppo(args, rank, world, local, device)
+    dist.barrier()
+    dist.destroy_process_group()
+
+
+if __name__ == "__main__":
+    main()

code/train/ctrl_policy.py

@@ -0,0 +1,389 @@
+"""Controller policy of the decoupled plan (PLAN_DECOUPLED_20261013.md sections 3 and 5): a frozen perception gives
+(p, h, v) of the current target, a small MLP on the last 20 steps picks the MineRL action. Runs in the eval venv
+(numpy, openai; no torch). MineExplorer/eval_benchmark.py (MC_POLICY) and the GRPO rollouts call Policy.act once per
+step; train/ctrl.py (torch) trains the MLP and exports it as npz: w0, b0, w1, b1, w2, b2 with weights as (in, out).
+
+Actions, three heads sampled independently:
+- movement, 15 combinations (MOVES); sprint only with forward, else MineExplorer's validation swaps in a no-op. A VPT
+  key bitmask maps to its class by cancelling opposite keys (as the game does), then dropping sneak, sprint and jump
+  in turn until the combination is one of MOVES;
+- camera yaw, 17 bins (YAWS: 0, +-0.5 ... +-64 deg, > 0 turns right); camera pitch, 13 bins (PITCHES: 0, +-0.5 ...
+  +-16 deg, > 0 looks down). A recorded delta maps to the nearest bin on a log scale, 0 only for an exact 0.
+attack, use, inventory, hotbar, ESC and the remaining keys are always 0.
+Input: 5 numbers per step, the last 20 steps oldest first (100): p, h/HALF_HFOV, v/HALF_VFOV of the current target
+(bench/diag_perception.py view: h > 0 right of the view center, v > 0 above it) and the policy's own previous camera
+action yaw/64, pitch/16. Steps before the episode start are 0; when the target changes, the target part of the
+earlier steps is set to 0. A controller whose w0 has 116 rows also gets 16 memory inputs after the window (memory():
+for the 8 directions yaw + 45 i, i = 0..7, whether the direction's world-yaw sector was a view sector and whether the
+4x4 cell 4 blocks along it was stood in, over the episode's poses up to the current one).
+Perception:
+- gt: p = 1 and the exact h, v when the target is in the geometric field of view (|h| <= 51.2, |v| <= 35), else 0;
+- vlm: two questions, sent at the same time, on the frame the eval client sends (320x180 PNG), thinking off,
+  temperature 0, worded as measured: step 1's grounding question (bench/diag_perception_direction.py GROUND,
+  max_tokens 48) with the answer started as START, whose first box's center through the pinhole model gives h, v
+  (0 without a box), and step 1b's presence score s_yes (bench/diag_perception_presence.py YESNO, max_tokens 1:
+  P(yes) / (P(yes) + P(no)) from the top 20 logprobs) as p. Asked on the episode's first step, when the target name
+  changes and PERCEIVE_EVERY steps after the last call; the steps in between carry the last answer over (carry()).
+Current target: the first milestone in order that has rules, was not satisfied at spawn and is not completed (the
+checker marks it); its name is the milestone task "find X" -> X with underscores as spaces.
+Per-step log (Policy.act): x (the input, rounded to 4 decimals; the forward pass uses exactly these values), target
+(milestone index), bins [move, yaw, pitch], logp (sum over the three heads), perception (vlm: reply, box, p on the
+steps that asked, None on the carried ones; gt: None), held (steps since the last VLM call: 0 on the steps that asked,
+and always for gt), truth of the current target (seen = in the geometric field of view, dist from the feet as the
+milestone check, h, v; the checker's yaw error is -h).
+"""
+from collections import deque
+from concurrent.futures import ThreadPoolExecutor
+import math
+import time
+from pathlib import Path
+import sys
+
+import numpy as np
+from openai import OpenAI
+
+ROOT = Path(__file__).resolve().parents[1]
+sys.path.append(str(ROOT / "bench"))
+from diag_perception import EYE, HALF_VFOV, view  # noqa: E402
+from diag_perception_direction import BOX, GROUND, TAN_HALF_HFOV  # noqa: E402
+from diag_perception_presence import YESNO  # noqa: E402
+
+WINDOW = 20
+START = '```json\n[\n\t{"bbox_2d": ['  # the start of the grounding answer (ground())
+PERCEIVE_EVERY = 3  # vlm: steps from one perception call to the next (find25 evals and GRPO rollouts)
+HALF_HFOV = math.degrees(math.atan(TAN_HALF_HFOV))  # 51.2
+MOVES = [(), ("forward",), ("forward", "jump"), ("forward", "right"), ("forward", "left"), ("forward", "sprint"),
+         ("forward", "jump", "sprint"), ("back",), ("right",), ("left",), ("back", "left"), ("back", "right"),
+         ("forward", "right", "jump"), ("jump",), ("forward", "left", "jump")]
+YAWS = np.array([-64, -32, -16, -8, -4, -2, -1, -0.5, 0, 0.5, 1, 2, 4, 8, 16, 32, 64])
+PITCHES = np.array([-16, -8, -4, -2, -1, -0.5, 0, 0.5, 1, 2, 4, 8, 16])
+CAM_SCALE = np.array([16.0, 64.0])  # camera [pitch, yaw] -> input
+KEYS = {"forward": 1, "back": 2, "left": 4, "right": 8, "jump": 16, "sprint": 32, "sneak": 64}  # VPT key bitmask
+NOOP = {"ESC": 0, "attack": 0, "back": 0, "forward": 0, "left": 0, "right": 0, "jump": 0, "camera": [0.0, 0.0],
+        "drop": 0, **{f"hotbar.{i}": 0 for i in range(1, 10)}, "inventory": 0, "pickItem": 0, "sneak": 0,
+        "sprint": 0, "swapHands": 0, "use": 0}
+
+
+def move_class(keys):
+    """MOVES index of a VPT key bitmask."""
+    masks = [sum(KEYS[k] for k in m) for m in MOVES]
+    for pair in (3, 12):  # forward+back, left+right
+        if keys & pair == pair:
+            keys &= ~pair
+    for drop in (0, 64, 32, 16):
+        keys &= ~drop
+        if keys in masks:
+            return masks.index(keys)
+
+
+MOVE_OF_KEYS = np.array([move_class(k) for k in range(128)])
+
+
+def cam_bin(deg, bins):
+    """Bin index (into YAWS or PITCHES) of camera deltas in degrees."""
+    j = np.clip(np.round(np.log2(np.maximum(np.abs(deg), 1e-9) / 0.5)), 0, len(bins) // 2 - 1)
+    return (len(bins) // 2 + np.sign(deg) * (j + 1)).astype(int)
+
+
+def gt_features(h, v):
+    """GT perception (..., 3): p, h/HALF_HFOV, v/HALF_VFOV in the geometric field of view, else 0."""
+    seen = (np.abs(h) <= HALF_HFOV) & (np.abs(v) <= HALF_VFOV)
+    return np.stack([seen, h / HALF_HFOV * seen, v / HALF_VFOV * seen], -1).astype(float)
+
+
+def rows(target, prev_cam):
+    """Window steps (..., 5) from target features (..., 3) and the previous camera action [pitch, yaw] (..., 2)."""
+    return np.concatenate([target, (prev_cam / CAM_SCALE)[..., ::-1]], -1)
+
+
+def sector(yaw):
+    """World-yaw sector of a yaw in degrees as logged (unwrapped too): 8 sectors of 45 deg, sector 0 = -22.5..22.5."""
+    return int(((yaw + 22.5) % 360) // 45)
+
+
+def cell(x, z):
+    """4x4 XZ cell of a position, as bench/find_metrics.py."""
+    return math.floor(x / 4), math.floor(z / 4)
+
+
+def memory(sectors, cells, pose):
+    """The 16 memory inputs at pose: for the directions t = yaw + 45 i, i = 0..7 (> 0 turns right), 1 if sector(t) is
+    in sectors, then 1 if the cell 4 blocks along t (horizontal direction dx = -sin t, dz = cos t) is in cells."""
+    yaws = [pose["yaw"] + 45 * i for i in range(8)]
+    return np.array([sector(t) in sectors for t in yaws]
+                    + [cell(pose["x"] - 4 * math.sin(math.radians(t)), pose["z"] + 4 * math.cos(math.radians(t)))
+                       in cells for t in yaws], dtype=float)
+
+
+def logits(w, x):
+    """(move, yaw, pitch) logits for inputs x (..., 100, or 116 with memory); w = the exported npz."""
+    z = np.maximum(x @ w["w0"] + w["b0"], 0)
+    z = np.maximum(z @ w["w1"] + w["b1"], 0)
+    return np.split(z @ w["w2"] + w["b2"], [len(MOVES), len(MOVES) + len(YAWS)], -1)
+
+
+def log_softmax(z):
+    z = z - z.max(-1, keepdims=True)
+    return z - np.log(np.exp(z).sum(-1, keepdims=True))
+
+
+def action(move, yaw, pitch):
+    """MineRL action dict of the three bins."""
+    a = dict(NOOP, camera=[float(PITCHES[pitch]), float(YAWS[yaw])])
+    a.update({key: 1 for key in MOVES[move]})
+    return a
+
+
+def target_name(task):
+    """Name asked about a milestone task: "find X" -> X with underscores as spaces (diagnoses B, C and step 1)."""
+    return task[5:].removeprefix("a ").replace("_", " ")
+
+
+def current_target(checker, presatisfied):
+    """(milestone index, absolute target [x, y, z], name) of the current target."""
+    for i, m in enumerate(checker._milestones):
+        if m.get("rules") and m["milestone_id"] not in presatisfied and not checker._completed[i]:
+            params = m["rules"][0]["params"]
+            target = params["target"]
+            if params.get("coordinate_frame") == "spawn_relative":
+                o = checker._spawn_pos
+                target = [target[0] + o["x"], target[1] + o["y"], target[2] + o["z"]]
+            return i, target, target_name(m["task"])
+
+
+def box_angles(box):
+    """(h, v) of a box [x1, y1, x2, y2] in Qwen's relative 0-1000 coordinates: its center through the pinhole model
+    (exact at pitch 0), as step 1."""
+    x1, y1, x2, y2 = box
+    return (math.degrees(math.atan(((x1 + x2) / 1000 - 1) * TAN_HALF_HFOV)),
+            math.degrees(math.atan((1 - (y1 + y2) / 1000) * math.tan(math.radians(HALF_VFOV)))))
+
+
+def chat(client, model, frame, prompt, max_tokens, start=None, **kwargs):
+    """One question about one frame (base64 PNG), thinking off, temperature 0, as steps 1 and 1b asked it. start: the
+    beginning of the answer, which the model continues (the reply is the continuation, its stop string included)."""
+    messages = [{"role": "user", "content": [
+        {"type": "image_url", "image_url": {"url": "data:image/png;base64," + frame}},
+        {"type": "text", "text": prompt}]}]
+    extra = {"chat_template_kwargs": {"enable_thinking": False}}
+    if start:
+        messages.append({"role": "assistant", "content": start})
+        extra.update(continue_final_message=True, add_generation_prompt=False, include_stop_str_in_output=True)
+    return client.chat.completions.create(model=model, temperature=0, max_tokens=max_tokens, messages=messages,
+                                          extra_body=extra, **kwargs).choices[0]
+
+
+def ground(client, model, frame, name):
+    """Step 1's grounding question, max_tokens 48 -> (reply, first box or None, h, v); h = v = 0 without a box.
+    The answer is started as START, how 15,505 of the first 15,533 replies of the BC find25 began (the other 28, 20
+    of them about melon, said there is none and now get a box), so the model writes the first box's four numbers and
+    stops at the first "]"; the prompt is then the question's followed by the tokens the model wrote for START
+    (vLLM /tokenize). reply: START and the model's numbers."""
+    reply = START + chat(client, model, frame, GROUND.format(name), 48, start=START, stop=["]"]).message.content
+    box = BOX.search(reply)
+    if not box:
+        return reply, None, 0.0, 0.0
+    box = [float(c) for c in box.groups()]
+    return reply, box, *box_angles(box)
+
+
+def presence(client, model, frame, name):
+    """Step 1b's s_yes: diagnosis B's yes/no question, max_tokens 1, top 20 logprobs; P(yes) / (P(yes) + P(no)) with
+    case and leading-space variants summed."""
+    top = chat(client, model, frame, YESNO.format(name), 1, logprobs=True, top_logprobs=20).logprobs.content[0]
+    p = {w: sum(math.exp(t.logprob) for t in top.top_logprobs if t.token.strip().lower() == w) for w in ("yes", "no")}
+    return p["yes"] / (p["yes"] + p["no"])
+
+
+def vlm_features(client, model, frame, name):
+    """VLM perception -> (p, h/HALF_HFOV, v/HALF_VFOV), log {reply, box, p}; the two questions are sent at once."""
+    with ThreadPoolExecutor(1) as pool:
+        grounding = pool.submit(ground, client, model, frame, name)
+        p = presence(client, model, frame, name)
+        reply, box, h, v = grounding.result()
+    return np.array([p, h / HALF_HFOV, v / HALF_VFOV]), {"reply": reply, "box": box, "p": round(p, 4)}
+
+
+def carry(p, box, yaw, pitch, pose):
+    """Features at pose of a VLM answer (p, first box or None) given at view yaw, pitch, as the C_vlm simulation keeps
+    a box's world point (train/ctrl.py simulate()): h moves by -delta yaw, v by +delta pitch, the box center stays in
+    the image; p is held, and without a box h = v = 0."""
+    if box is None:
+        return np.array([p, 0.0, 0.0])
+    h, v = box_angles(box)
+    h = np.clip(h - ((pose["yaw"] - yaw + 180) % 360 - 180), -HALF_HFOV, HALF_HFOV)
+    v = np.clip(v + pose["pitch"] - pitch, -HALF_VFOV, HALF_VFOV)
+    return np.array([p, h / HALF_HFOV, v / HALF_VFOV])
+
+
+class Policy:
+    """The controller for one episode. seed: anything np.random.default_rng takes (eval: [seed, task id]);
+    perception "gt" or "vlm" (url and model of the perception server)."""
+
+    def __init__(self, npz, seed, perception="gt", url=None, model=None):
+        self.w = dict(np.load(npz))
+        self.rng = np.random.default_rng(seed)
+        self.perception, self.model = perception, model
+        self.client = OpenAI(api_key="EMPTY", base_url=url) if perception == "vlm" else None
+        self.window = deque((np.zeros(5) for _ in range(WINDOW)), maxlen=WINDOW)
+        self.target, self.cam = None, [0.0, 0.0]
+        self.asked, self.held = None, 0  # vlm: (name, p, box, yaw, pitch) of the last call; steps since it
+        self.memory = len(self.w["w0"]) == 5 * WINDOW + 16
+        self.sectors, self.cells = set(), set()  # memory: view sectors and cells of the episode's poses so far
+
+    def vlm_features(self, frame, name):
+        """The VLM perception call (train/optimus_policy.py ExpertPerceptionPolicy overrides it)."""
+        return vlm_features(self.client, self.model, frame, name)
+
+    def act(self, frame, info, checker, presatisfied):
+        """frame: this step's frame as the eval client sends it (base64 PNG, 320x180; used by vlm); info: the env
+        info before the action (player_pos); checker: the episode's MilestoneChecker; presatisfied: ids of milestones
+        satisfied at spawn. Returns (MineRL action dict, step log)."""
+        index, target, name = current_target(checker, presatisfied)
+        if index != self.target:
+            for r in self.window:
+                r[:3] = 0
+            self.target = index
+        pose = info["player_pos"]
+        dist, h, v = view(pose, target)
+        if self.perception == "gt":
+            feat, raw = gt_features(h, v), None
+        elif self.asked and self.asked[0] == name and self.held < PERCEIVE_EVERY - 1:
+            self.held += 1
+            feat, raw = carry(*self.asked[1:], pose), None
+        else:
+            feat, raw = self.vlm_features(frame, name)
+            self.asked, self.held = (name, feat[0], raw["box"], pose["yaw"], pose["pitch"]), 0
+        self.window.append(rows(feat, np.array(self.cam)))
+        x = np.concatenate(self.window)
+        if self.memory:
+            self.sectors.add(sector(pose["yaw"]))
+            self.cells.add(cell(pose["x"], pose["z"]))
+            x = np.concatenate([x, memory(self.sectors, self.cells, pose)])
+        x = np.round(x, 4)
+        heads = [log_softmax(z) for z in logits(self.w, x)]
+        bins = [int(self.rng.choice(len(lp), p=np.exp(lp))) for lp in heads]
+        a = action(*bins)
+        self.cam = a["camera"]
+        return a, {"x": x.tolist(), "target": index, "bins": bins,
+                   "logp": float(sum(lp[b] for lp, b in zip(heads, bins))), "perception": raw, "held": self.held,
+                   "truth": {"seen": bool(gt_features(h, v)[0]), "dist": round(dist, 2), "h": round(h, 2),
+                             "v": round(v, 2)}}
+
+
+# BT arm (EXPERIMENT_LOG.md 2026-10-17): the VLM (Qwen3.5-9B + LoRA) reads the current frame and the target name and
+# writes a behavior code, one digit per FSQ dimension of train/bt_tokenizer.py; the next CHUNK ticks run open loop,
+# each tick's three bins sampled from the decoder's distributions for the code (table.npz probs, mixed radix index).
+CHUNK = 20
+BT_PROMPT = "Task: find {}. Reply with the behavior code for the next second."
+BT_MEMORY = " Seen directions (0, 45, ..., 315 deg right of view): {}. Visited cells 4 blocks along them: {}."
+BT_NOOP = [0, len(YAWS) // 2, len(PITCHES) // 2]
+
+
+def bt_prompt(name, mem=None):
+    """The BT text prompt for target name; with mem (the two 8-character 0/1 strings of memory(), directions then
+    cells) BT_MEMORY follows."""
+    return BT_PROMPT.format(name) + (BT_MEMORY.format(*mem) if mem else "")
+
+
+def bt_code(reply, levels):
+    """Table index of a reply: its first len(levels) characters as digits, each clipped to its level; None if they
+    are not all digits."""
+    digits = reply[:len(levels)]
+    if len(digits) < len(levels) or not digits.isdigit():
+        return None
+    index = 0
+    for d, level in zip(digits, levels):
+        index = index * level + min(int(d), level - 1)
+    return index
+
+
+class BTPolicy:
+    """The BT policy for one episode, with Policy's act(). table: the tokenizer's decode-table npz; url, model: the
+    vLLM serving the LoRA. seed: None (GRPO rollouts) or [eval seed, task id]; decision k of the episode then samples
+    with seed eval_seed * 10**7 + task_id * 1000 + k, and the tick bins with a numpy Generator seeded by
+    eval_seed * 10**7 + task_id * 1000. Asks on the episode's first step, every CHUNK ticks and when the current target
+    changes; an unparseable reply runs CHUNK no-op ticks (valid False). memory True (the curiosity variant,
+    EXPERIMENT_LOG.md 2026-10-18): every step adds the pose's sector and cell to the episode's sets as Policy does, and
+    each decision's prompt carries memory() at that pose as two 0/1 strings (bt_prompt()).
+    Per-step log: target, bins, tick (index in the chunk), truth as Policy; on decision steps also reply, code (the
+    clipped digits), valid, logprobs (vLLM, per generated token), act_s (seconds of the call) and, with memory, memory
+    (the two strings in the prompt)."""
+
+    def __init__(self, table, seed, url, model, memory=False):
+        t = np.load(table)
+        self.levels, self.probs = [int(x) for x in t["levels"]], t["probs"]
+        self.seed = None if seed is None else seed[0] * 10**7 + seed[1] * 1000
+        self.rng = np.random.default_rng(self.seed)
+        self.client, self.model = OpenAI(api_key="EMPTY", base_url=url), model
+        self.target, self.tick, self.decisions, self.chunk = None, CHUNK, 0, None
+        self.memory, self.sectors, self.cells = memory, set(), set()
+
+    def ask(self, frame, name, mem=None):
+        extra = {"chat_template_kwargs": {"enable_thinking": False}}
+        if self.seed is not None:
+            extra["seed"] = self.seed + self.decisions
+        choice = self.client.chat.completions.create(
+            model=self.model, temperature=1.0, max_tokens=len(self.levels), logprobs=True, extra_body=extra,
+            messages=[{"role": "user", "content": [
+                {"type": "image_url", "image_url": {"url": "data:image/png;base64," + frame}},
+                {"type": "text", "text": bt_prompt(name, mem)}]}]).choices[0]
+        return choice.message.content, [t.logprob for t in choice.logprobs.content]
+
+    def act(self, frame, info, checker, presatisfied):
+        index, target, name = current_target(checker, presatisfied)
+        pose = info["player_pos"]
+        dist, h, v = view(pose, target)
+        log = {"target": index}
+        if self.memory:
+            self.sectors.add(sector(pose["yaw"]))
+            self.cells.add(cell(pose["x"], pose["z"]))
+        if index != self.target or self.tick == CHUNK:
+            t = time.monotonic()
+            mem = None
+            if self.memory:
+                bits = "".join(str(int(b)) for b in memory(self.sectors, self.cells, pose))
+                mem = log["memory"] = [bits[:8], bits[8:]]
+            reply, logprobs = self.ask(frame, name, mem)
+            code = bt_code(reply, self.levels)
+            self.chunk = self.probs[code] if code is not None else None
+            self.target, self.tick = index, 0
+            self.decisions += 1
+            log.update(reply=reply, valid=code is not None, logprobs=logprobs, act_s=round(time.monotonic() - t, 3),
+                       code=code is not None and "".join(str(min(int(d), l - 1)) for d, l in zip(reply, self.levels))
+                       or None)
+        bins = BT_NOOP if self.chunk is None else [
+            int(self.rng.choice(len(p), p=p / p.sum()))
+            for p in np.split(self.chunk[self.tick].astype(float), [len(MOVES), len(MOVES) + len(YAWS)])]
+        log.update(bins=bins, tick=self.tick, truth={"seen": bool(gt_features(h, v)[0]), "dist": round(dist, 2),
+                                                     "h": round(h, 2), "v": round(v, 2)})
+        self.tick += 1
+        return action(*bins), log
+
+
+class NoopPolicy:
+    """Every step the no-op action (kind "noop")."""
+
+    def act(self, frame, info, checker, presatisfied):
+        return dict(NOOP), {}
+
+
+def make_policy(kind="ctrl", **kwargs):
+    """Policy (kind "ctrl") or BTPolicy (kind "bt") from an MC_POLICY / rollout job policy dict; kinds "hid" and
+    "hid_teacher": the hidden-state arms' policies (train/hid_policy.py)."""
+    if kind.startswith("hid"):
+        from hid_policy import make
+        return make(kind, **kwargs)
+    if kind == "box":  # task 4b (EXPERIMENT_LOG.md 2026-10-19): train/box_policy.py
+        from box_policy import BoxPolicy
+        return BoxPolicy(**kwargs)
+    if kind == "noop":  # what a 1B expert that never writes parseable JSON amounts to (EXPERIMENT_LOG.md 2026-10-19)
+        return NoopPolicy()
+    if kind == "openha":  # OpenHA baseline (EXPERIMENT_LOG.md 2026-10-19): train/openha_policy.py
+        from openha_policy import OpenHAPolicy
+        return OpenHAPolicy(**kwargs)
+    if kind.startswith("optimus"):  # task 1A (EXPERIMENT_LOG.md 2026-10-19): train/optimus_policy.py
+        import optimus_policy
+        return (optimus_policy.OptimusActionPolicy if kind == "optimus_action"
+                else optimus_policy.ExpertPerceptionPolicy)(**kwargs)
+    return BTPolicy(**kwargs) if kind == "bt" else Policy(**kwargs)

code/train/grpo_locations.json

@@ -0,0 +1 @@
+{"0004": [3630.5, 70.0, 1967.5], "0005": [4163.5, 63.0, 1937.5], "0007": [4990.5, 68.0, 1967.5], "0008": [5390.5, 67.0, 1967.5], "0012": [6990.5, 64.0, 1967.5], "0017": [9150.5, 68.0, 1967.5], "0022": [11150.5, 63.0, 1967.5], "0024": [11630.5, 79.0, 1967.5], "0025": [12190.5, 70.0, 1967.5], "0039": [4830.5, 64.0, 2367.5], "0045": [7390.5, 88.0, 2367.5], "0051": [9630.5, 68.0, 2367.5], "0053": [10430.5, 88.0, 2367.5], "0064": [2030.5, 63.0, 2767.5], "0071": [4830.5, 74.0, 2767.5], "0072": [5390.5, 69.0, 2767.5], "0075": [6430.5, 64.0, 2767.5], "0077": [7230.5, 95.0, 2767.5], "0083": [9618.5, 70.0, 2806.5], "0099": [3390.5, 80.0, 3167.5], "0100": [3950.5, 87.0, 3167.5], "0101": [4190.5, 79.0, 3167.5], "0114": [9230.5, 64.0, 3167.5], "0126": [14350.5, 70.0, 3167.5], "0131": [3390.5, 81.0, 3567.5], "0132": [3950.5, 67.0, 3567.5], "0135": [5150.5, 79.0, 3567.5], "0144": [8430.5, 64.0, 3567.5], "0153": [11985.5, 68.0, 3662.5], "0157": [13630.5, 66.0, 3567.5], "0167": [5150.5, 64.0, 3967.5], "0174": [7630.5, 63.0, 3967.5], "0187": [12785.5, 78.0, 4062.5], "0190": [14030.5, 67.0, 3967.5], "0204": [6830.5, 64.0, 4367.5], "0205": [7550.5, 70.0, 4367.5], "0214": [10830.5, 69.0, 4367.5], "0215": [11390.5, 63.0, 4367.5], "0218": [12590.5, 77.0, 4367.5], "0222": [13985.5, 68.0, 4462.5], "0226": [2785.5, 94.0, 4862.5], "0227": [3230.5, 70.0, 4767.5], "0228": [3630.5, 67.0, 4767.5], "0229": [3985.5, 67.0, 4862.5], "0245": [10545.5, 73.0, 4862.5], "0248": [11790.5, 87.0, 4767.5], "0250": [12430.5, 111.0, 4767.5], "0256": [2030.5, 68.0, 5167.5], "0257": [2430.5, 71.0, 5167.5], "0258": [2785.5, 63.0, 5262.5], "0264": [5230.5, 64.0, 5167.5], "0268": [6830.5, 76.0, 5167.5], "0269": [7390.5, 83.0, 5167.5], "0275": [9950.5, 65.0, 5167.5], "0277": [10430.5, 68.0, 5167.5], "0278": [10990.5, 73.0, 5167.5], "0283": [12830.5, 70.0, 5167.5], "0288": [2350.5, 77.0, 5567.5], "0290": [2830.5, 71.0, 5567.5], "0293": [4350.5, 79.0, 5567.5], "0301": [7230.5, 89.0, 5567.5], "0302": [7790.5, 74.0, 5567.5], "0306": [9550.5, 69.0, 5567.5], "0307": [9790.5, 68.0, 5567.5], "0325": [4030.5, 74.0, 5967.5], "0335": [8063.5, 68.0, 5998.5], "0340": [10350.5, 70.0, 5967.5], "0344": [11790.5, 66.0, 5967.5], "0350": [13985.5, 65.0, 6062.5], "0351": [14463.5, 76.0, 5998.5], "0354": [2830.5, 71.0, 6367.5], "0355": [3230.5, 82.0, 6367.5], "0361": [5709.5, 68.0, 6416.5], "0364": [7023.5, 64.0, 6398.5], "0369": [9150.5, 69.0, 6367.5], "0374": [10830.5, 70.0, 6367.5], "0387": [3230.5, 69.0, 6767.5], "0402": [9230.5, 65.0, 6767.5], "0415": [14371.5, 63.0, 6752.5], "0424": [5218.5, 70.0, 7206.5], "0429": [7465.5, 66.0, 7181.5], "0433": [8990.5, 72.0, 7167.5], "0435": [9630.5, 70.0, 7167.5], "0436": [10190.5, 77.0, 7167.5], "0438": [10830.5, 69.0, 7167.5], "0440": [11790.5, 75.0, 7167.5], "0443": [12830.5, 67.0, 7167.5], "0445": [13745.5, 63.0, 7262.5], "0452": [3630.5, 92.0, 7567.5], "0454": [4418.180300335881, 67.0, 7606.180300335881], "0467": [9790.5, 75.0, 7567.5], "0469": [10430.5, 73.0, 7567.5], "0474": [12590.5, 64.0, 7567.5], "0478": [14030.5, 86.0, 7567.5], "0479": [14750.5, 67.0, 7567.5], "0483": [3390.5, 67.0, 7967.5], "0484": [3790.5, 69.0, 7967.5], "0486": [4578.5, 66.0, 8006.5], "0494": [7630.5, 86.0, 7967.5], "0497": [9150.5, 67.0, 7967.5], "0503": [11230.5, 80.0, 7967.5], "0509": [13709.5, 63.0, 8016.5], "0515": [3550.5, 70.0, 8367.5], "0516": [3630.5, 64.0, 8367.5], "0521": [5950.5, 74.0, 8367.5], "0523": [6430.5, 81.0, 8367.5], "0524": [6830.5, 75.0, 8367.5], "0528": [8430.5, 66.0, 8367.5], "0544": [2350.5, 72.0, 8767.5], "0550": [4430.5, 72.0, 8767.5], "0551": [4830.5, 74.0, 8767.5], "0553": [5630.5, 63.0, 8767.5], "0554": [6030.5, 83.0, 8767.5], "0558": [7630.5, 76.0, 8767.5], "0569": [12030.5, 73.0, 8767.5], "0572": [13345.5, 75.0, 8862.5], "0576": [2350.5, 68.0, 9167.5], "0577": [2430.5, 63.0, 9167.5], "0578": [2990.5, 64.0, 9167.5], "0585": [5790.5, 64.0, 9167.5], "0593": [9065.5, 67.0, 9181.5], "0598": [10990.5, 68.0, 9167.5], "0608": [2030.5, 69.0, 9567.5], "0618": [6030.5, 76.0, 9567.5], "0620": [6978.5, 70.0, 9606.5], "0625": [8830.5, 70.0, 9567.5], "0628": [10350.5, 65.0, 9567.5], "0631": [11230.5, 70.0, 9567.5], "0632": [11745.5, 66.0, 9662.5], "0635": [12990.5, 64.0, 9567.5], "0637": [13630.5, 63.0, 9567.5], "0639": [14430.5, 72.0, 9567.5], "0640": [2190.5, 72.0, 9967.5], "0645": [4030.5, 63.0, 9967.5], "0648": [5230.5, 80.0, 9967.5], "0652": [6830.5, 73.0, 9967.5], "0667": [12830.5, 68.0, 9967.5], "0671": [14430.5, 73.0, 9967.5], "0672": [2030.5, 89.0, 10367.5], "0676": [3630.5, 70.0, 10367.5], "0677": [4190.5, 71.0, 10367.5], "0681": [5630.5, 80.0, 10367.5], "0684": [6990.5, 74.0, 10367.5], "0694": [10990.5, 70.0, 10367.5], "0697": [12350.5, 85.0, 10367.5], "0700": [13390.5, 85.0, 10367.5], "0703": [14738.5, 63.0, 10406.5], "0704": [2030.5, 78.0, 10767.5], "0705": [2430.5, 73.0, 10767.5], "0710": [4590.5, 79.0, 10767.5], "0712": [5390.5, 63.0, 10767.5], "0714": [6190.5, 64.0, 10767.5], "0722": [9390.5, 70.0, 10767.5], "0731": [13150.5, 69.0, 10767.5], "0741": [4030.5, 68.0, 11167.5], "0747": [6418.5, 63.0, 11206.5], "0756": [10145.5, 69.0, 11262.5], "0765": [13630.5, 71.0, 11167.5], "0767": [14750.5, 64.0, 11167.5], "0770": [2990.5, 91.0, 11567.5], "0771": [3390.5, 66.0, 11567.5], "0775": [4830.5, 71.0, 11567.5], "0788": [10190.5, 74.0, 11567.5], "0790": [10818.5, 77.0, 11606.5], "0792": [11950.5, 64.0, 11567.5], "0797": [13790.5, 69.0, 11567.5], "0800": [2030.5, 67.0, 11967.5], "0801": [2430.5, 67.0, 11967.5], "0802": [2830.5, 72.0, 11967.5], "0807": [5150.5, 67.0, 11967.5], "0808": [5230.5, 72.0, 11967.5]}

code/train/hid_policy.py

@@ -0,0 +1,101 @@
+"""Policies of the hidden-state arms (EXPERIMENT_LOG.md 2026-10-19, tasks 3 and 4a); eval venv (numpy, requests).
+
+HidPolicy (MC_POLICY / rollout kind "hid"): the controller's input is [feature (4096), camera history (40)] -> MLP
+(train/hid_bc.py Net: 256 -> 256 -> A's three heads, exported as ctrl_policy's npz layout, read by ctrl_policy.logits).
+- feature: train/hid_server.py's Qwen3.5-9B hidden state for (the current frame, the current target name), asked on
+  A's perception schedule (the episode's first step, when the target name changes, PERCEIVE_EVERY steps after the
+  last call) and held unchanged in between;
+- camera history: A's window camera part, the previous camera action [yaw/64, pitch/16] of the last 20 steps oldest
+  first (zeros before the episode start; not reset on a target change), rounded to 4 decimals as A's x.
+Sampling as ctrl_policy.Policy (three heads, numpy Generator seeded by seed). url: the feature server (a list: one
+picked per process). Options: log_feat logs the feature (base64 bf16 bits) on asking steps; frames marks asking
+steps with "frame" (train/grpo_rollout.py then saves the PNG); ref (an npz) asks the server for the base feature too
+and logs ref, the 45 log-probs of that controller on [base feature, camera history] (the 4a KL reference).
+Per-step log: target, bins, logp, cam (the 40 history values), held (0 on asking steps), feat_s on asking steps
+(seconds of the call), truth as Policy; plus feat / frame / ref as above.
+
+Teacher (kind "hid_teacher"): A's controller (ctrl_policy.Policy, gt perception) whose log marks with "frame" the
+steps where A's vlm schedule would ask (the BC demonstrations, train/hid_bc.py collect).
+"""
+import base64
+from collections import deque
+import os
+import time
+
+import numpy as np
+import requests
+
+from ctrl_policy import (CAM_SCALE, PERCEIVE_EVERY, WINDOW, Policy, action, current_target, gt_features, log_softmax,
+                         logits, view)
+
+
+def decode(s):
+    """base64 bf16 bits -> float32 vector (exact)."""
+    return (np.frombuffer(base64.b64decode(s), np.uint16).astype(np.uint32) << 16).view(np.float32)
+
+
+class HidPolicy:
+    def __init__(self, npz, seed, url, log_feat=False, frames=False, ref=None):
+        self.w = dict(np.load(npz))
+        self.ref = dict(np.load(ref)) if ref else None
+        self.rng = np.random.default_rng(seed)
+        urls = url if isinstance(url, list) else [url]
+        self.url = urls[os.getpid() % len(urls)]
+        self.log_feat, self.frames = log_feat, frames
+        self.cams = deque((np.zeros(2) for _ in range(WINDOW)), maxlen=WINDOW)
+        self.cam = [0.0, 0.0]
+        self.asked, self.held, self.feat, self.base = None, 0, None, None
+
+    def act(self, frame, info, checker, presatisfied):
+        index, target, name = current_target(checker, presatisfied)
+        pose = info["player_pos"]
+        dist, h, v = view(pose, target)
+        log = {}
+        if self.asked == name and self.held < PERCEIVE_EVERY - 1:
+            self.held += 1
+        else:
+            t = time.monotonic()
+            r = requests.post(self.url + "/feat", json={"frame": frame, "name": name}, timeout=600)
+            r.raise_for_status()
+            r = r.json()
+            self.feat = decode(r["feat"])
+            self.base = decode(r["base"]) if self.ref else None
+            self.asked, self.held = name, 0
+            log["feat_s"] = round(time.monotonic() - t, 3)
+            if self.log_feat:
+                log["feat"] = r["feat"]
+            if self.frames:
+                log["frame"] = True
+        self.cams.append(np.array(self.cam)[::-1] / CAM_SCALE[::-1])
+        cam = np.round(np.concatenate(self.cams), 4)
+        heads = [log_softmax(z) for z in logits(self.w, np.concatenate([self.feat, cam]))]
+        bins = [int(self.rng.choice(len(lp), p=np.exp(lp))) for lp in heads]
+        a = action(*bins)
+        self.cam = a["camera"]
+        if self.ref:
+            log["ref"] = np.round(np.concatenate([log_softmax(z) for z in logits(self.ref, np.concatenate(
+                [self.base, cam]))]), 4).tolist()
+        log.update(target=index, bins=bins, logp=float(sum(lp[b] for lp, b in zip(heads, bins))), cam=cam.tolist(),
+                   held=self.held, truth={"seen": bool(gt_features(h, v)[0]), "dist": round(dist, 2),
+                                          "h": round(h, 2), "v": round(v, 2)})
+        return a, log
+
+
+class Teacher(Policy):
+    def __init__(self, npz, seed):
+        super().__init__(npz, seed, "gt")
+        self.asked_name, self.since = None, 0
+
+    def act(self, frame, info, checker, presatisfied):
+        name = current_target(checker, presatisfied)[2]
+        a, log = super().act(frame, info, checker, presatisfied)
+        if self.asked_name == name and self.since < PERCEIVE_EVERY - 1:
+            self.since += 1
+        else:
+            self.asked_name, self.since = name, 0
+            log["frame"] = True
+        return a, log
+
+
+def make(kind, **kwargs):
+    return Teacher(**kwargs) if kind == "hid_teacher" else HidPolicy(**kwargs)

code/train/hid_server.py

@@ -0,0 +1,151 @@
+"""Hidden-state feature of the hidden-state arms (EXPERIMENT_LOG.md 2026-10-19, tasks 3 and 4a) and its HTTP server.
+Runs in .train-venv.
+
+Feature: Qwen3.5-9B's last-layer hidden state (after the final norm, i.e. the lm_head input; 4096 dims) at the last
+prompt token of A's grounding question (ctrl_policy.ground(): the frame, GROUND.format(name), thinking off, the answer
+started as START, rendered by the chat template with continue_final_message as vLLM renders it), prefill only. Batches
+are right-padded and each row's last real token is gathered. Values travel as base64 of the bf16 bits (encode/decode;
+hid_policy.decode is the numpy twin), so a logged feature is exactly the one the controller used.
+
+Server: one model replica per GPU (threads), a shared queue, each replica takes up to MAX_BATCH waiting requests per
+forward. POST /feat {"frame": base64 PNG, "name": target name} -> {"feat": ...} and, with --base, also "base": the
+feature with the adapter disabled (the 4a KL reference input). The port opens once every replica is loaded.
+Usage: CUDA_VISIBLE_DEVICES=2 .train-venv/bin/python train/hid_server.py --port 9210 [--gpus 0] [--adapter DIR --base]
+"""
+import argparse
+import base64
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+import io
+import json
+from pathlib import Path
+import queue
+import sys
+import threading
+import time
+
+import torch
+from PIL import Image
+from transformers import AutoProcessor
+
+from bt_train import MODULES, load_model  # noqa: F401  (MODULES: the LoRA targets, re-exported for hid_grpo)
+from ctrl_policy import START
+
+ROOT = Path(__file__).resolve().parents[1]
+sys.path.append(str(ROOT / "bench"))
+from diag_perception_direction import GROUND  # noqa: E402
+
+MAX_BATCH = 32
+
+
+def processor():
+    return AutoProcessor.from_pretrained(ROOT / "models/Qwen3.5-9B", local_files_only=True)
+
+
+def prompt(proc, name):
+    return proc.apply_chat_template(
+        [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": GROUND.format(name)}]},
+         {"role": "assistant", "content": START}],
+        tokenize=False, add_generation_prompt=False, continue_final_message=True, enable_thinking=False)
+
+
+def batch_inputs(proc, images, names):
+    return dict(proc(text=[prompt(proc, n) for n in names], images=images, padding=True, padding_side="right",
+                     return_tensors="pt"))
+
+
+def features(model, inputs, device):
+    """(B, 4096) bf16 last-layer hidden states at each row's last prompt token; model: the Qwen3.5 model or its
+    PeftModel (grad follows the caller's mode)."""
+    body = (model.get_base_model() if hasattr(model, "get_base_model") else model).model
+    x = {k: v.to(device) for k, v in inputs.items()}
+    with torch.autocast("cuda", dtype=torch.bfloat16):
+        h = body(**x, use_cache=False).last_hidden_state
+    return h[torch.arange(len(h), device=device), x["attention_mask"].sum(1) - 1]
+
+
+def encode(f):
+    """bf16 vector -> base64 of its bits."""
+    return base64.b64encode(f.to(torch.bfloat16).view(torch.int16).cpu().numpy().tobytes()).decode()
+
+
+def decode(s):
+    return torch.frombuffer(bytearray(base64.b64decode(s)), dtype=torch.int16).view(torch.bfloat16)
+
+
+def png(b64):
+    return Image.open(io.BytesIO(base64.b64decode(b64))).convert("RGB")
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--port", type=int, required=True)
+    ap.add_argument("--gpus", default="0", help="local cuda indices, one replica each")
+    ap.add_argument("--adapter", type=Path)
+    ap.add_argument("--base", action="store_true", help="also return the feature with the adapter disabled")
+    args = ap.parse_args()
+    proc = processor()
+    jobs = queue.Queue()
+
+    def replica(model, device):
+        while True:
+            items = [jobs.get()]
+            while len(items) < MAX_BATCH:
+                try:
+                    items.append(jobs.get_nowait())
+                except queue.Empty:
+                    break
+            try:
+                inputs = batch_inputs(proc, [png(i["frame"]) for i in items], [i["name"] for i in items])
+                with torch.no_grad():
+                    out = {"feat": features(model, inputs, device)}
+                    if args.base:
+                        with model.disable_adapter():
+                            out["base"] = features(model, inputs, device)
+                for k, item in enumerate(items):
+                    item["out"] = {key: encode(f[k]) for key, f in out.items()}
+            except Exception as e:  # answer every waiting request rather than leave it hanging
+                for item in items:
+                    item["out"] = {"error": f"{type(e).__name__}: {e}"}
+            for item in items:
+                item["done"].set()
+
+    gpus = [int(g) for g in args.gpus.split(",")]
+    for g in gpus:
+        model = load_model(g)
+        if args.adapter:
+            from peft import PeftModel
+            model = PeftModel.from_pretrained(model, args.adapter)
+        model.eval()
+        threading.Thread(target=replica, args=(model, torch.device("cuda", g)), daemon=True).start()
+
+    class Handler(BaseHTTPRequestHandler):
+        def log_message(self, *a):
+            pass
+
+        def reply(self, code, body):
+            data = json.dumps(body).encode()
+            self.send_response(code)
+            self.send_header("Content-Type", "application/json")
+            self.send_header("Content-Length", str(len(data)))
+            self.end_headers()
+            self.wfile.write(data)
+
+        def do_GET(self):
+            self.reply(200, {"gpus": gpus})
+
+        def do_POST(self):
+            item = json.loads(self.rfile.read(int(self.headers["Content-Length"])))
+            item["done"] = threading.Event()
+            jobs.put(item)
+            item["done"].wait()
+            self.reply(500 if "error" in item["out"] else 200, item["out"])
+
+    server = ThreadingHTTPServer(("0.0.0.0", args.port), Handler)
+    server.daemon_threads = True
+    print(f"{time.strftime('%Y-%m-%d %H:%M:%S')} feature server on port {args.port}, gpus {gpus}, adapter "
+          f"{args.adapter}, base {args.base}", flush=True)
+    server.serve_forever()
+
+
+if __name__ == "__main__":
+    main()

patches/MineExplorer.patch

@@ -0,0 +1,437 @@
+diff --git a/env/minerl_sandbox.py b/env/minerl_sandbox.py
+index b74f0e8..b3f3860 100644
+--- a/env/minerl_sandbox.py
++++ b/env/minerl_sandbox.py
+@@ -194,7 +194,8 @@ class MineRLSandboxEnv(gym.Env):
+         for attempt in range(max_retries):
+             try:
+                 logger.info(f"Resetting environment (attempt {attempt + 1}/{max_retries}) session_id={self.session_id}...")
+-                response = self._post("reset_env", timeout=180)
++                # a client timeout must not trigger a retry while the server is still resetting (two resets at once)
++                response = self._post("reset_env", timeout=int(os.getenv("MC_RESET_TIMEOUT", "180")))
+                 if response.get("status") != 0:
+                     raise RuntimeError(f"Server error during reset: {response.get('msg')}")
+                 return self._get_obs_from_response(response), {}
+diff --git a/eval_benchmark.py b/eval_benchmark.py
+index 54487bd..a579b48 100644
+--- a/eval_benchmark.py
++++ b/eval_benchmark.py
+@@ -22,6 +22,10 @@ if str(_ROOT_DIR) not in sys.path:
+ from env.minerl_sandbox import MineRLSandboxEnv
+ from env.render import RenderWrapper
+ from mc_agent import DefaultAgent, MinerRLActionSpace, OpenAIProvider, VLLMProvider, DefaultContextBuilder
++from mc_agent.utils import convert_buffer_to_base64_images
++
++sys.path.append(str(_ROOT_DIR.parent / "train"))
++from ctrl_policy import make_policy  # noqa: E402
+ 
+ FRAME_BUFFER_SIZE = 20
+ # MAX_STEPS = 50
+@@ -45,13 +49,22 @@ class MineRLBenchmarkEnv(MineRLSandboxEnv):
+             raise FileNotFoundError(f"Metadata not found: {meta_path}")
+         with open(meta_path, "r", encoding="utf-8") as f:
+             self._metadata = json.load(f)
++        self._scenario_id = meta_path.parent.parent.name
+         self._parse_metadata()
+         super().__init__(env_id=self._scene_name, use_friday=use_friday)
+ 
+     def _parse_metadata(self) -> None:
+-        self._commands_list = self._metadata.get("commands", [])
++        # command feedback ("Changed the block at ...") covered the lower-left of the first ~150 frames
++        self._commands_list = ["/gamerule sendCommandFeedback false"] + self._metadata.get("commands", [])
+         self._task_text = self._metadata.get("task_text", "")
+         self._scene_name = self._metadata.get("scene_name", "benchmark_scene")
++        self._world_seed = 0
++        if os.getenv("MC_EXPLORE_TASK"):  # goal-free exploration: bare world, one fixed seed per scenario id
++            self._commands_list = ["/gamerule sendCommandFeedback false", "/time set day"]
++            self._task_text = os.environ["MC_EXPLORE_TASK"]
++            self._world_seed = int(self._scenario_id) + 1
++        if os.getenv("MC_START_YAW"):  # start facing this yaw (the reversed-start find25 variant: 180)
++            self._commands_list.append(f"/tp @p ~ ~ ~ {os.environ['MC_START_YAW']} 0")
+         logger.info(f"[BenchmarkEnv] scene={self._scene_name} commands={len(self._commands_list)}")
+         logger.info(f"[BenchmarkEnv] task: {self._task_text[:100]}")
+ 
+@@ -61,7 +74,7 @@ class MineRLBenchmarkEnv(MineRLSandboxEnv):
+             env='MinecraftSim',
+             obs_size=[128, 128],
+             render_size=[640, 360],
+-            seed=0,
++            seed=self._world_seed,
+             record=False,
+             record_path='./output/',
+             yaml_config=None,
+@@ -118,6 +131,7 @@ def _run_benchmark(
+     vllm_url: str = "http://localhost:8000/v1",
+     frame_size: int = FRAME_BUFFER_SIZE,
+     use_friday: bool = False,
++    end_on_esc: bool = False,
+ ) -> Dict[str, Any]:
+     """Run one benchmark scenario and save results."""
+     global _shutdown_requested
+@@ -129,8 +143,15 @@ def _run_benchmark(
+     scene_id = meta_path.parent.parent.name
+     task_desc = metadata_dict.get("task_text", f"Complete the Minecraft task for {scene_id}.")
+     safe_model = model.replace("/", "_")
+-
+-    if _HAS_MILESTONE_CHECKER:
++    explore_task = os.getenv("MC_EXPLORE_TASK")  # goal-free: no milestones, always runs max_steps
++    # MC_POLICY (JSON: npz, perception gt|vlm, url and model of the perception server for vlm, seed) runs the
++    # controller of train/ctrl_policy.py instead of agent.get_action; it samples with the seed and the scene id
++    policy_cfg = json.loads(os.getenv("MC_POLICY", "null"))
++    policy = policy_cfg and make_policy(**{**policy_cfg, "seed": [policy_cfg["seed"], int(scene_id)]})
++    if explore_task:
++        task_desc = explore_task
++        checker = MilestoneChecker([])
++    elif _HAS_MILESTONE_CHECKER:
+         try:
+             checker = MilestoneChecker.from_metadata(metadata_path)
+             logger.info(f"Loaded {len(checker._milestones)} milestones")
+@@ -159,13 +180,16 @@ def _run_benchmark(
+ 
+     env = RenderWrapper(_base_env, save_messages=True, save_path=str(output_dir))
+ 
+-    frame_buffer = deque(maxlen=frame_size)
++    # MC_FRAME_STRIDE=k feeds frame_size frames spaced k steps apart (default 1: consecutive).
++    frame_stride = int(os.getenv("MC_FRAME_STRIDE", "1"))
++    agent.frame_stride = frame_stride
++    frame_buffer = deque(maxlen=(frame_size - 1) * frame_stride + 1)
+     thought_history: List[str] = []
+     action_history: List[Dict] = []
+     _frame_completed: Dict[str, Optional[int]] = {}
+     _presatisfied_ids: set = set()
+     _frame_offset: int = 0
+-    _all_done_logged: bool = False
++    initial_player_pos: Dict[str, Any] = {}
+     long_term_memory: str = ""
+ 
+     original_sigint_handler = signal.signal(signal.SIGINT, _signal_handler)
+@@ -173,7 +197,6 @@ def _run_benchmark(
+     try:
+         has_loading = loading_command_steps > 0
+         obs, info = env.reset(save_frame=not has_loading)
+-        frame_buffer.append(obs["pov"])
+ 
+         for step in range(loading_command_steps):
+             if _shutdown_requested:
+@@ -190,8 +213,9 @@ def _run_benchmark(
+         if not info.get("player_pos"):
+             _, noop_action = agent.get_default_action(is_call_failed=False)
+             obs, reward, terminated, truncated, info = env.step(noop_action, save_frame=True)
+-            frame_buffer.append(obs["pov"])
+ 
++        frame_buffer.append(obs["pov"])
++        initial_player_pos = dict(info.get("player_pos", {}))
+         checker.reset(info)
+         agent.load_system_prompt(task_desc)
+ 
+@@ -205,7 +229,8 @@ def _run_benchmark(
+         # agent takes any action).  These are treated as "pre-satisfied" and
+         # will NOT be counted as completed — they are excluded via a blocklist
+         # so that _frame_completed is only set when the agent genuinely
+-        # triggers the condition during gameplay.
++        # triggers the condition during gameplay. They must also be excluded
++        # from the required targets, otherwise the episode can never succeed.
+         _baseline_check = checker.check(info)
+         _presatisfied_ids: set = set()
+         for _bms in _baseline_check:
+@@ -217,11 +242,17 @@ def _run_benchmark(
+                 )
+         # Reset the checker state so those pre-satisfied flags don't persist
+         checker.reset(info)
++        _trackable = [
++            m["milestone_id"] for m in checker._milestones
++            if m.get("rules") and m["milestone_id"] not in _presatisfied_ids
++        ]
++        if not _trackable:
++            logger.info(f"[{run_id}] No unsatisfied trackable targets at spawn; excluding episode from scores.")
+ 
+         step_error: Optional[str] = None
+         step = -1
+ 
+-        for step in range(max_steps):
++        for step in range(max_steps if _trackable or explore_task else 0):
+             if _shutdown_requested:
+                 logger.info(f"[{run_id}] Shutdown requested, exiting after step {step}...")
+                 step_error = "interrupted_by_user"
+@@ -230,10 +261,17 @@ def _run_benchmark(
+             logger.info(f"[{run_id}] --- Step {step + 1}/{max_steps} ---")
+ 
+             try:
+-                thought, action, memory_update = agent.get_action(
+-                    list(frame_buffer), list(thought_history), list(action_history), step + 1,
+-                    long_term_memory=long_term_memory
+-                )
++                if policy:
++                    # a policy with raw_frame (train/optimus_policy.py) gets the env's RGB frame, not the 320x180 PNG
++                    frame = obs["pov"] if getattr(policy, "raw_frame", False) else \
++                        convert_buffer_to_base64_images([obs["pov"]])[0]
++                    action, policy_log = policy.act(frame, info, checker, _presatisfied_ids)
++                    thought = memory_update = ""
++                else:
++                    thought, action, memory_update = agent.get_action(
++                        list(frame_buffer)[::-1][::frame_stride][::-1], list(thought_history), list(action_history),
++                        step + 1, long_term_memory=long_term_memory
++                    )
+             except Exception as agent_err:
+                 logger.error(f"[{run_id}] Agent call failed: {agent_err}. Retrying in 10s...")
+                 time.sleep(10)
+@@ -249,6 +287,7 @@ def _run_benchmark(
+ 
+             thought_history.append(thought)
+             action_history.append(action)
++            player_pos_before = dict(info.get("player_pos", {}))
+ 
+             try:
+                 augmented_action = checker.augment_action_with_queries(action, info)
+@@ -258,6 +297,13 @@ def _run_benchmark(
+                 time.sleep(10)
+                 continue
+ 
++            with open(output_dir / "steps.jsonl", "a", encoding="utf-8") as _f:
++                _f.write(json.dumps({"step": step + 1, "thought": thought, "memory": long_term_memory,
++                                     "action": {k: v for k, v in action.items() if v not in (0, [0.0, 0.0], [0, 0])},
++                                     "player_pos_before": player_pos_before,
++                                     "player_pos_after": info.get("player_pos", {}),
++                                     **({"policy": policy_log} if policy else {})},
++                                    ensure_ascii=False) + "\n")
+             done = terminated or truncated
+             frame_buffer.append(obs["pov"])
+ 
+@@ -272,11 +318,17 @@ def _run_benchmark(
+                     _frame_completed[mid] = env.frame_count - _frame_offset
+                     logger.success(f"[{run_id}] Milestone '{mid}' completed at step {step + 1} (frame {_frame_completed[mid]})")
+ 
+-            if not _all_done_logged and checker.num_completed() > 0 and checker.all_done():
+-                _all_done_logged = True
+-                logger.success(f"[{run_id}] All milestones completed at step {step + 1}!")
++            if _trackable and all(m in _frame_completed for m in _trackable):
++                logger.success(f"[{run_id}] Every trackable milestone done at step {step + 1}, stopping early.")
++                break
++
++            # the prompt says ESC ends the episode, but the released sandbox never reports done for it and
++            # agents then press ESC for the rest of the budget; --end-on-esc ends it as the prompt promises
++            if end_on_esc and action.get("ESC"):
++                logger.info(f"[{run_id}] Agent pressed ESC at step {step + 1}, ending the episode.")
++                break
+ 
+-            if (step + 1) % 10 == 0:
++            if (step + 1) % 300 == 0:
+                 env.save_video_checkpoint()
+ 
+             if done:
+@@ -314,11 +366,12 @@ def _run_benchmark(
+         })
+ 
+     # Compute milestones_completed / all_milestones_done using only the
+-    # corrected _frame_completed dict (excludes pre-satisfied milestones).
++    # corrected _frame_completed dict. Pre-satisfied milestones are excluded
++    # from both the completed count and the number of required targets.
+     trackable_mids = {
+         ms.get("milestone_id", "")
+         for ms in checker._milestones
+-        if len(ms.get("rules", [])) > 0
++        if len(ms.get("rules", [])) > 0 and ms.get("milestone_id", "") not in _presatisfied_ids
+     }
+     corrected_completed = sum(
+         1 for mid in trackable_mids
+@@ -338,6 +391,9 @@ def _run_benchmark(
+         "mode": "multi-agent",
+         "metadata_path": str(meta_path),
+         "total_steps": total_steps,
++        "frame_size": frame_size,
++        "frame_stride": frame_stride,
++        "initial_player_pos": initial_player_pos,
+         "milestones_completed": corrected_completed,
+         "milestones_trackable": corrected_trackable,
+         "milestones_total": len(checker._milestones),
+@@ -358,6 +414,8 @@ def _run_benchmark(
+ def _worker_eval(worker_args: dict) -> dict:
+     import signal as _signal
+     _signal.signal(_signal.SIGINT, _signal.SIG_IGN)
++    if worker_args.get("sandbox_url"):
++        os.environ["MC_SANDBOX_URL"] = worker_args["sandbox_url"]
+ 
+     _root = Path(__file__).resolve().parent
+     sys.path.append(str(_root))
+@@ -372,6 +430,7 @@ def _worker_eval(worker_args: dict) -> dict:
+         vllm_url=worker_args.get("vllm_url", "http://localhost:8000/v1"),
+         frame_size=worker_args.get("frame_size", FRAME_BUFFER_SIZE),
+         use_friday=worker_args.get("use_friday", False),
++        end_on_esc=worker_args.get("end_on_esc", False),
+     )
+ 
+ 
+@@ -382,10 +441,12 @@ def eval_benchmark(
+                                       help="Path to benchmark directory"),
+     output_dir: str = typer.Option("outputs", "--output-dir", "-o",
+                                    help="Output directory"),
+-    loading_command_steps: int = typer.Option(20, "--loading-command-steps",
++    loading_command_steps: int = typer.Option(200, "--loading-command-steps",  # failed-command chat lines fade after 200 ticks
+                                               help="No-op steps after reset"),
+     max_steps: int = typer.Option(MAX_STEPS, "--max-steps",
+                                   help="Maximum agent steps per episode"),
++    frame_size: int = typer.Option(FRAME_BUFFER_SIZE, "--frame-size", min=1,
++                                   help="Number of recent observation frames"),
+     resume: bool = typer.Option(False, "--resume",
+                                 help="Resume evaluation (skip completed)"),
+     use_vllm: bool = typer.Option(False, "--use-vllm",
+@@ -398,6 +459,10 @@ def eval_benchmark(
+                                         help="Limit number of scenarios to evaluate"),
+     use_friday: bool = typer.Option(False, "--use-friday",
+                                     help="Use Friday platform sandbox instead of local Docker"),
++    tasks: Optional[str] = typer.Option(None, "--tasks",
++                                        help="Comma-separated scenario ids to evaluate (default: all)"),
++    end_on_esc: bool = typer.Option(False, "--end-on-esc",
++                                    help="End the episode when the agent presses ESC"),
+ ):
+     """Evaluate all benchmark scenarios in benchmark_dir."""
+     logger.info(f"--- Starting evaluation (model={model}) ---")
+@@ -418,6 +483,10 @@ def eval_benchmark(
+         logger.warning(f"No metadata.json found under {bench_path}")
+         return
+ 
++    if tasks:
++        wanted = set(tasks.split(","))
++        metadata_entries = [e for e in metadata_entries if e[0] in wanted]
++
+     if limit and limit > 0:
+         metadata_entries = metadata_entries[:limit]
+ 
+@@ -433,7 +502,18 @@ def eval_benchmark(
+         if resume and result_path.exists():
+             logger.info(f"[SKIP] {scene_num} - result.json exists")
+             with open(result_path, "r", encoding="utf-8") as f:
+-                all_results.append(json.load(f))
++                summary = json.load(f)
++            if not summary.get("error"):
++                milestones = MilestoneChecker.from_metadata(str(meta_path))._milestones
++                trackable = {m["milestone_id"] for m in milestones if m.get("rules")}
++                statuses = summary["milestone_status"]
++                trackable -= {m["milestone_id"] for m in statuses if m.get("presatisfied_at_spawn")}
++                summary["milestones_trackable"] = len(trackable)
++                summary["milestones_completed"] = sum(
++                    m["completed"] for m in statuses if m["milestone_id"] in trackable
++                )
++                summary["all_milestones_done"] = bool(trackable) and summary["milestones_completed"] == len(trackable)
++            all_results.append(summary)
+         else:
+             pending.append((scene_num, meta_path))
+ 
+@@ -466,9 +546,11 @@ def eval_benchmark(
+                     output_dir=scene_out_dir,
+                     loading_command_steps=loading_command_steps,
+                     max_steps=max_steps,
++                    frame_size=frame_size,
+                     use_vllm=use_vllm,
+                     vllm_url=vllm_url,
+                     use_friday=use_friday,
++                    end_on_esc=end_on_esc,
+                 )
+             except Exception as e:
+                 logger.error(f"[ERROR] {scene_num}: {e}")
+@@ -485,51 +567,44 @@ def eval_benchmark(
+     else:
+         import multiprocessing as _mp
+ 
+-        effective_workers = min(num_workers, len(pending))
+         mp_ctx = _mp.get_context("spawn")
+-
+-        remaining = list(pending)
+-        batch_num = 0
+-
+-        while remaining:
+-            batch = remaining[:effective_workers]
+-            remaining = remaining[effective_workers:]
+-            batch_num += 1
+-
+-            logger.info(f"\n[Batch {batch_num}] Launching {len(batch)} worker(s)")
+-
+-            batch_jobs: List[Dict[str, Any]] = []
+-            for scene_num, meta_path in batch:
+-                scene_out_dir = out_root / scene_num
+-                scene_out_dir.mkdir(parents=True, exist_ok=True)
+-                batch_jobs.append({
+-                    "metadata_path": str(meta_path),
+-                    "model": model,
+-                    "output_dir": str(scene_out_dir),
+-                    "loading_command_steps": loading_command_steps,
+-                    "max_steps": max_steps,
+-                    "use_vllm": use_vllm,
+-                    "vllm_url": vllm_url,
+-                    "use_friday": use_friday,
+-                    "_scene_num": scene_num,
+-                })
+-
+-            with ProcessPoolExecutor(max_workers=len(batch_jobs), mp_context=mp_ctx) as pool:
+-                futures = {pool.submit(_worker_eval, job): job for job in batch_jobs}
+-                for future in as_completed(futures):
+-                    job = futures[future]
+-                    scene_num = job["_scene_num"]
+-                    try:
+-                        summary = future.result()
+-                        all_results.append(summary)
+-                        task_icon = "✅" if summary.get("all_milestones_done") else "❌"
+-                        logger.info(f"[DONE] {scene_num} {task_icon} | milestones {summary.get('milestones_completed', 0)}/{summary.get('milestones_trackable', 0)}")
+-                    except Exception as exc:
+-                        logger.error(f"[ERROR] {scene_num}: {exc}")
+-                        summary = _make_error_summary(scene_num, exc)
+-                        all_results.append(summary)
+-
+-    total = len(all_results)
++        urls = [u for u in os.getenv("MC_SANDBOX_URLS", "").split(",") if u]
++        jobs: List[Dict[str, Any]] = []
++        for i, (scene_num, meta_path) in enumerate(pending):
++            scene_out_dir = out_root / scene_num
++            scene_out_dir.mkdir(parents=True, exist_ok=True)
++            jobs.append({
++                "metadata_path": str(meta_path),
++                "model": model,
++                "output_dir": str(scene_out_dir),
++                "loading_command_steps": loading_command_steps,
++                "max_steps": max_steps,
++                "frame_size": frame_size,
++                "use_vllm": use_vllm,
++                "vllm_url": vllm_url,
++                "use_friday": use_friday,
++                "end_on_esc": end_on_esc,
++                "sandbox_url": urls[i % len(urls)] if urls else None,
++                "_scene_num": scene_num,
++            })
++
++        with ProcessPoolExecutor(max_workers=min(num_workers, len(jobs)), mp_context=mp_ctx) as pool:
++            futures = {pool.submit(_worker_eval, job): job for job in jobs}
++            for future in as_completed(futures):
++                job = futures[future]
++                scene_num = job["_scene_num"]
++                try:
++                    summary = future.result()
++                    all_results.append(summary)
++                    task_icon = "✅" if summary.get("all_milestones_done") else "❌"
++                    logger.info(f"[DONE] {scene_num} {task_icon} | milestones {summary.get('milestones_completed', 0)}/{summary.get('milestones_trackable', 0)}")
++                except Exception as exc:
++                    logger.error(f"[ERROR] {scene_num}: {exc}")
++                    summary = _make_error_summary(scene_num, exc)
++                    all_results.append(summary)
++
++    scored_results = [r for r in all_results if r.get("milestones_trackable", 0) > 0 or r.get("error")]
++    total = len(scored_results)
+     done = sum(1 for r in all_results if r.get("all_milestones_done"))
+     ms_done = sum(r.get("milestones_completed", 0) for r in all_results)
+     ms_total = sum(r.get("milestones_trackable", 0) for r in all_results)
+@@ -539,7 +614,7 @@ def eval_benchmark(
+     logger.info("-" * 60)
+     task_rate = done / total * 100 if total > 0 else 0.0
+     ms_rate = ms_done / ms_total * 100 if ms_total > 0 else 0.0
+-    logger.info(f"Scenes: {total} | Tasks: {done}/{total} ({task_rate:.1f}%) | Milestones: {ms_done}/{ms_total} ({ms_rate:.1f}%)")
++    logger.info(f"Scenes: {len(all_results)} | Eligible tasks: {done}/{total} ({task_rate:.1f}%) | Milestones: {ms_done}/{ms_total} ({ms_rate:.1f}%)")
+     logger.info(f"{'='*60}")
+ 
+     agg_path = out_root / "eval_summary.json"
+@@ -548,6 +623,7 @@ def eval_benchmark(
+             "model": model,
+             "benchmark": str(bench_path),
+             "total": total,
++            "excluded": len(all_results) - total,
+             "done": done,
+             "ms_done": ms_done,
+             "ms_total": ms_total,

patches/sandbox_mc_server.patch

@@ -0,0 +1,52 @@
+--- a/mc_server.py
++++ b/mc_server.py
+@@ -118,6 +118,30 @@
+     raise RuntimeError("obs/info contains neither 'image' nor 'pov'")
+ 
+ 
++_INFO_SKIP = {"pov", "image", "location_stats", "equipped_items", "custom"}
++
++
++def info_for_client(info: dict) -> dict:
++    """Judge-relevant fields of the sim info (the released server returned none)."""
++    out = {k: convert_numpy_types(dict(v) if isinstance(v, dict) else v)
++           for k, v in (info or {}).items() if k not in _INFO_SKIP}
++    inv = {}
++    for slot in (info or {}).get("inventory", {}).values():
++        if slot.get("type") not in (None, "none", "air"):
++            inv[slot["type"]] = inv.get(slot["type"], 0) + int(slot.get("quantity", 0))
++    out["inventory"] = inv
++    raw = (info or {}).get("inventory", {})
++    hotbar = []
++    for i in range(9):
++        slot = raw.get(i) or raw.get(str(i)) or {}
++        t = slot.get("type")
++        hotbar.append(None if t in (None, "none", "air") else f"{t} x{int(slot.get('quantity', 0))}")
++    out["hotbar"] = hotbar
++    hand = (info or {}).get("equipped_items", {}).get("mainhand", {})
++    out["mainhand"] = hand.get("type") if isinstance(hand, dict) else None
++    return out
++
++
+ # =========================
+ # API 实现
+ # =========================
+@@ -224,6 +248,10 @@
+         return err(f"session '{session_id}' not found")
+ 
+     try:
++        if session.env_type == "minestudio":
++            # MineRL clears its seed at the end of every reset (even a failed one), so a retried reset
++            # would build a random world; re-apply the create_env seed each time
++            session.env.env.seed(session.env.seed)
+         result = session.env.reset()
+ 
+         if isinstance(result, tuple):
+@@ -276,6 +304,7 @@
+             "screenshot": screenshot,
+             "reward": float(reward),
+             "done": done,
++            "info": info_for_client(info),
+             "timing": {
+                 "env_step_ms": round((t2 - t1) * 1000, 1),
+                 "encode_ms": round((t4 - t3) * 1000, 1),