Single <think>-prefilled prompt format, no-reasoning mode, full Doom example; drop benchmark presets

README.md

@@ -105,10 +105,10 @@ THUSI-Lab/GameScaling
 │   └── 27b_s3_1600h_gs7020/
 ├── prompts/
 │   ├── system_prompt_template.txt  standard prompt, 3 slots (§2)
-│   └── example_doom_battle1.txt    filled-in example
+│   └── example_doom_battle1.txt    complete Doom prompt (§2)
 ├── gamescaling/                inference client
 │   ├── prompt.py               system prompt builders
-│   ├── policy.py               client, presets, request body
+│   ├── policy.py               client and request body
 │   ├── action.py               action format and parser
 │   └── keymaps.py              key names <-> browser keys
 ├── scripts/serve_vllm.sh       start a vLLM server
@@ -162,145 +162,185 @@ Things to keep as shipped:
 - **One server per GPU group.** One vLLM replica can serve several games at once, but long
   prefills (one image per history turn) make each step slower when many games share it.
 
-## 2. System prompt template
+## 2. System prompt
 
 `prompts/system_prompt_template.txt` is the standard system prompt. Fill in three placeholders
-and keep everything else **exactly as written**: the model decides its output tags from the
-exact text of the Output Format section.
+and keep everything else exactly as written: the model decides its output tags from the exact
+text of the Output Format section.
 
-| Placeholder | What to write | Example |
-|---|---|---|
-| `{GAME_NAME}` | The game's name. It appears twice: in the opening and the key-list heading. | `Doom Battle-1` |
-| `{GAME_RULES}` | One English paragraph: what the game is, the goal, and how it is scored. | `Kill as many monsters as possible; collect health and ammo.` |
-| `{KEY_BINDINGS}` | One `- <key> -> <effect>` line per key. Use training key names (§5). | see below |
+| Placeholder | What to write |
+|---|---|
+| `{GAME_NAME}` | The game's name. It appears twice: in the opening and in the key-list heading. |
+| `{GAME_RULES}` | What the game is, what is on screen, the goal and how it is scored. Plain text; several lines are fine. |
+| `{KEY_BINDINGS}` | One `- <key> -> <effect>` line per key, in training key names (§5). End with the mouse line below if the mouse turns the camera. |
+
+The mouse line, copied verbatim:
+
+```text
+- Mouse X Y -> aim / rotate the camera (X>0 right, Y>0 down)
+```
+
+> **Note.** Lines such as `**Game Rules**`, `**Output Format**` and
+> `**Explanation**` are plain-text section headers that the model saw in training. They are part
+> of the prompt, not Markdown formatting: send them to the model unchanged, including the
+> asterisks.
+
+### Complete example: Doom Battle-1
 
-Example `{KEY_BINDINGS}`:
+This is the full system prompt for the ViZDoom Battle-1 map, as in
+`prompts/example_doom_battle1.txt`. It is the same text in both reasoning modes (§4).
 
 ```text
+You are a gaming expert, and you are currently playing Doom Battle-1. You are proficient in keyboard and mouse operations. You understand game mechanics and combat pacing, can quickly extract key information from the current screen, and can think and make precise decisions at critical moments. Based on the current screen, plan the next 200ms of actions, consisting of 6 steps spaced 33ms apart. Each step lasts 33ms, until the next step begins. If the current situation continues the previous strategy, output the action directly. Think only when the situation changes significantly, the previous analysis is no longer valid, or a new objective appears.
+**Game Rules**
+You are playing Doom, a first-person shooter, in an arena full of monsters. Survive and kill as many monsters as you can.
+
+## What you see on screen
+- First-person view.
+- The crosshair at screen centre marks where shots land.
+- The status bar along the bottom shows HEALTH on the left and AMMO on the right.
+- Monsters keep arriving for the whole episode; some throw fireballs from range.
+
+## Rules
+- Score is the number of monsters you kill in the episode.
+- Firing costs ammo, and the episode ends as soon as your health reaches 0, so dodging matters as much as shooting: standing still in the open is how most episodes end early.
+- Health packs and ammo boxes lie around the map; walk over one to pick it up. Running out of either is fatal, so collect them while fighting.
+- A target is hit when it is under the crosshair, so turn with the mouse until the monster is centred before firing.
+- The map is one large open arena, so monsters can close in from any direction.
+**Common Keys in Doom Battle-1**
 - W -> Move forward
-- S -> Move backward
 - A -> Strafe left
+- S -> Move backward
 - D -> Strafe right
-- LB -> Fire
+- LB -> Fire weapon
+- Ctrl -> Fire weapon (keyboard alternative)
+- Shift -> Run
+- Space -> Open doors / use switches
+- one -> Select weapon slot 1
+- two -> Select weapon slot 2
+- three -> Select weapon slot 3
 - Mouse X Y -> aim / rotate the camera (X>0 right, Y>0 down)
+**Output Format**
+**Action only (usual)**
+<think></think><|action_start|>X Y Z ; k1 k2 k3 ; k4 k5 ; k6 ; k7 ; k8 ; k9 k10<|action_end|>
+**Thinking + action (when necessary)**
+<think>reasoning</think><|action_start|>X Y Z ; k1 k2 k3 ; k4 k5 ; k6 ; k7 ; k8 ; k9 k10<|action_end|>
+**Explanation**
+1. **Mouse Movement**: First, specify the relative displacement X, Y (X>0 means move right, Y>0 means move down) and scroll amount Z (Z>0 means scroll up).
+2. **Key Sequence**: Then list 6 groups of keys; within each group, keys are separated by spaces, and groups are separated by semicolons.
+- Each group can contain up to 4 keys.
+- If a group has no keys, leave it empty but keep the `;`.
+3. Only output a plain string that conforms to the above format — no line breaks and no quotation marks.
+**Key Naming Rules**
+- Number keys `0-9`: use lowercase English words, e.g., `zero` for `0`, `one` for `1`, ... `nine` for `9`.
+- Function keys `F1-F12`: use capitalized English words, e.g., `One` represents `F1`, `Two` represents `F2`, and so on.
+- Mouse buttons: use `LB`, `RB`, `MB` for the left, right, and middle mouse button.
+- Letters: use the uppercase letter, e.g., `A`, `D`, `W`, `S`.
+- Arrow keys: use `Up`, `Down`, `Left`, `Right`.
+- Modifier/special keys: `Shift`, `Ctrl`, `Alt`, `Tab`, `Caps`, `Esc`, `Space`, `Enter`, `Back` (backspace), `Delete`, `Insert`, `Home`, `End`, `Pause`.
 ```
 
-Add the last line only for games where the mouse controls the camera, and copy it verbatim. Key
-names: uppercase letters (`W`); digits as `zero`..`nine`; F1-F12 as `One`..`Twelve`; mouse
-buttons as `LB` / `RB` / `MB`; arrows as `Up` / `Down` / `Left` / `Right`. A complete filled-in
-prompt is in `prompts/example_doom_battle1.txt`.
+The same prompt from code (`tests/test_core.py` checks that it matches the file byte for byte):
+
+```python
+from gamescaling import build_system_prompt
+
+DOOM_RULES = open("prompts/example_doom_battle1.txt").read() \
+    .split("**Game Rules**\n", 1)[1].split("\n**Common Keys in", 1)[0]
+
+system = build_system_prompt(
+    "Doom Battle-1",
+    rules=DOOM_RULES,
+    key_bindings=[
+        ("W", "Move forward"), ("A", "Strafe left"), ("S", "Move backward"), ("D", "Strafe right"),
+        ("LB", "Fire weapon"), ("Ctrl", "Fire weapon (keyboard alternative)"), ("Shift", "Run"),
+        ("Space", "Open doors / use switches"), ("one", "Select weapon slot 1"),
+        ("two", "Select weapon slot 2"), ("three", "Select weapon slot 3"),
+    ],
+    mouse_aim=True,   # appends the "Mouse X Y -> ..." line
+)
+```
 
-Using the text file directly:
+Filling the text file directly gives the same result:
 
 ```python
 tpl = open("prompts/system_prompt_template.txt").read().rstrip("\n")
 system = (tpl.replace("{GAME_NAME}", "Doom Battle-1")
-             .replace("{GAME_RULES}", "Kill as many monsters as possible; collect health and ammo.")
-             .replace("{KEY_BINDINGS}", "- W -> Move forward\n- LB -> Fire\n"
-                      "- Mouse X Y -> aim / rotate the camera (X>0 right, Y>0 down)"))
+             .replace("{GAME_RULES}", DOOM_RULES)
+             .replace("{KEY_BINDINGS}", "- W -> Move forward\n- A -> Strafe left\n..."
+                      "\n- Mouse X Y -> aim / rotate the camera (X>0 right, Y>0 down)"))
 ```
 
-The result is byte-identical to what `build_system_prompt(...)` below produces;
-`tests/test_core.py` checks this.
-
 ## 3. Minimal usage
 
 ```python
 from PIL import Image
-from gamescaling import GameScalingPolicy, PolicyConfig, build_system_prompt
+from gamescaling import GameScalingPolicy, PolicyConfig
 
-system = build_system_prompt(
-    "Doom Battle-1",
-    rules="Kill as many monsters as possible; collect health and ammo.",
-    key_bindings={"W": "Move forward", "S": "Move backward", "A": "Strafe left",
-                  "D": "Strafe right", "LB": "Fire"},
-    mouse_aim=True,               # the mouse controls the camera
-)                                 # reasoning=True by default, matching the <think> prefill
-policy = GameScalingPolicy(system, PolicyConfig())   # prefills <think>; frame sent at native size
+policy = GameScalingPolicy(system, PolicyConfig())   # system prompt from §2; prefills <think>
 policy.reset()                                       # call at the start of every episode
 decision = policy.act(Image.open("frame.png"))       # one call = one 200 ms decision
 a = decision.action
 a.mouse      # (X, Y, Z): mouse movement for this chunk (per-mille of the screen) + scroll
 a.steps      # 6 groups: the keys held during each 33 ms step
 a.parsed     # False = could not be parsed; execute as a no-op
+decision.reasoning   # the model's thinking for this step ("" if it acted directly)
 ```
 
 Pass `a` to your executor. Run 6 steps of 33 ms, holding that step's keys during each one. Keys
 that appear in two consecutive groups stay held rather than being pressed again. Mouse movement
 is `X/1000 * screen width` and `Y/1000 * screen height`. You can spread it over the 200 ms or
-send it at once.
-
-Defaults are not tied to any benchmark: any game frame, the standard system prompt, and a
-`<think>` prefill. To reproduce a paper table, use
-`PolicyConfig.preset("omnigamearena" | "vizdoom" | "gameworld")` instead (§6).
-`TRAINING_SYSTEM_PROMPT` is the generic pre-training prompt. Its Output Format is action-only and
-does not match a `<think>` prefill, so use it only for ablations.
-
-## 4. Prompt and message structure
+send it at once. The frame is sent at its native size; set `PolicyConfig(image_size=(w, h))` to
+resize it first.
 
-Every benchmark uses the same skeleton in a fixed order (`gamescaling/prompt.py`):
+## 4. Reasoning modes and message structure
 
-1. **Opening**: gaming expert + game name + the contract "plan the next 200 ms as 6 steps of
-   33 ms";
-2. `**Game Rules**`: what the game is and how it is scored;
-3. **Controls**: the keys the game uses, **written in the training vocabulary** (§5);
-4. `**Task Instruction**`: GameWorld only (one game can have several tasks);
-5. `**Output Format**`: how the model picks its output format;
-6. `**Explanation**` + `**Key Naming Rules**`.
+The assistant turn is always prefilled, and the request continues it
+(`continue_final_message=True`). The system prompt is the same in both modes.
 
-There are two layouts. They differ only in section 3 and the separators:
-
-- `build_system_prompt(...)`: key list (`**Common Keys in <game>**` + `- W -> Move forward`).
-  Used for ViZDoom, OmniGameArena and **any new game**. Add `mouse_aim=True` for games where the
-  mouse controls the camera.
-- `build_gameworld_system_prompt(...)`: free-text paragraphs (`**Role and Controls**`), for
-  games that come with their own written controls, as in GameWorld. `translate_controls`
-  rewrites browser key names into the training vocabulary (`ArrowLeft` -> `Left`, `a/d` ->
-  `A/D`, `1-9:` -> `one-nine:`).
-
-The Output Format **must match whether the client prefills `<think>`**:
-
-| `reasoning` | Client | Model output |
+| Mode | Prefill | What the model does |
 |---|---|---|
-| `True` (default) | `think_prefill=True`: the assistant turn starts with `<think>` | Decides whether and how long to think, closes `</think>`, then outputs the action |
-| `False` | no prefill | Outputs `<|action_start|>...<|action_end|>` directly |
+| `PolicyConfig()` (default, `reasoning=True`) | `<think>` | Decides whether and how long to think, closes `</think>`, then outputs the action |
+| `PolicyConfig(reasoning=False)` | `<think>\n\n</think>` | Thinking is closed and empty; the model outputs `<|action_start|>...<|action_end|>` directly |
+
+`<think>\n\n</think>` is exactly how the chat template renders a turn without reasoning, so the
+no-reasoning mode stays in the training format. Do not send the request without a prefill.
 
 Each user turn is the text `current game screen` followed by the frame. Multi-turn messages
 (`policy.py`):
 
-```
-system     system prompt
+```text
+system     system prompt (§2)
 user       "current game screen" + frame t-k          ┐ last history_len turns (default 5)
 assistant  action taken at t-k (no reasoning)          ┘
 ...
 user       "current game screen" + current frame
-assistant  "<think>"   (prefill; continue_final_message=True)
+assistant  "<think>"  or  "<think>\n\n</think>"       (prefill, continued by the model)
 ```
 
-Past turns keep only the action (`history_think=0`, as in the paper). A turn that failed to parse
-is stored as an explicit no-op, `<|action_start|>0 0 0 ; ; ; ; ; ; <|action_end|>`, not as the
-raw bad output.
+Past turns keep only the action (`history_think=0`). A turn that failed to parse is stored as an
+explicit no-op, `<|action_start|>0 0 0 ; ; ; ; ; ; <|action_end|>`, not as the raw bad output.
 
-Default request: `temperature=1.0`, `max_tokens=2048`, `stop=["<|action_end|>"]`,
+Client defaults: `temperature=1.0`, `max_tokens=2048`, `stop=["<|action_end|>"]`,
 `skip_special_tokens=false`, JPEG frames (quality 90) at native resolution.
 
 ## 5. Action format
 
-```
+```text
 <|action_start|>X Y Z ; g1 ; g2 ; g3 ; g4 ; g5 ; g6<|action_end|>
 ```
 
 - `X Y`: mouse movement over the whole 200 ms, in per-mille of the screen
   (`X = Σdx / screen width × 1000`), clipped to ±1000. X>0 is right, Y>0 is down.
 - `Z`: scroll notches, clipped to ±5. Z>0 is up.
-- `g1..g6`: six 33 ms steps. Each group lists the keys **held** during that step, at most 4. An
+- `g1..g6`: six 33 ms steps. Each group lists the keys held during that step, at most 4. An
   empty group releases all keys, but the `;` must stay.
 - `<|action_start|>` / `<|action_end|>` are special tokens. Requests must set
   `skip_special_tokens: false` (`build_request` does), or the server strips the tags.
   `parse_action` also accepts a bare `X Y Z ; ...` as a fallback.
-- The action is read **only after the last `</think>`**: a model that thinks often quotes the
-  format inside its reasoning. An unclosed `<think>` means the generation was cut off; it is
-  executed as a no-op (`Action.truncated=True`).
+- The action is read only after the last `</think>`: a model that thinks often quotes the format
+  inside its reasoning. An unclosed `<think>` means the generation was cut off; it is executed as
+  a no-op (`Action.truncated=True`).
 
 Training key vocabulary (74 tokens, case-sensitive, `gamescaling.action.KEY_VOCAB`):
 
@@ -308,64 +348,34 @@ Training key vocabulary (74 tokens, case-sensitive, `gamescaling.action.KEY_VOCA
 |---|---|
 | Letters | `A`..`Z` |
 | Number row | `zero` `one` .. `nine` |
-| F1-F12 | `One` `Two` .. `Twelve` (**capitalized** = function key) |
+| F1-F12 | `One` `Two` .. `Twelve` (capitalized = function key) |
 | Mouse buttons | `LB` `RB` `MB` |
 | Arrows | `Up` `Down` `Left` `Right` |
 | Other | `Shift` `Ctrl` `Alt` `Tab` `Caps` `Esc` `Space` `Enter` `Back` `Delete` `Insert` `Home` `End` `Pause` `Equal` `Minus` `Period` `Slash` `Quote` |
 
 Tokens outside the vocabulary are dropped and recorded in `Action.dropped`.
-`keymaps.TO_PLAYWRIGHT` maps them to browser key names.
-
-## 6. Evaluation
-
-The paper evaluates on four benchmarks: MCU (Minecraft), ViZDoom, OmniGameArena and GameWorld.
-To get comparable numbers, keep these settings fixed.
-
-**Clock.** The standard setting is **time-paused**: the game freezes while the model thinks and
-advances only while a chunk executes. In a real-time setting the game keeps running during
-inference, so latency becomes part of the score. Do not compare scores across clocks.
-
-**Client presets.** Each benchmark has a preset in `PolicyConfig.preset(...)`:
-
-| Preset | Frame | `<think>` prefill | Other |
-|---|---|---|---|
-| `omnigamearena` | resized to 854×480, JPEG | yes | 300 steps per episode (500 for `scene_escape` and `cue_chase`), 3 episodes |
-| `vizdoom` | resized to 854×480, JPEG | yes | no stop string; 4 maps × 10 episodes, at most 350 chunks each |
-| `gameworld` | height 480, PNG (Lanczos) | no | `max_tokens=4096`, lenient parsing; use `build_gameworld_system_prompt(..., reasoning=False)` |
-
-All presets keep 5 history turns (frame + action, no reasoning) at temperature 1.0. Run at least
-3 episodes per game; single episodes vary a lot.
-
-**Checklist before reading a score:**
-
-- [ ] The system prompt is the template from §2 (or `build_system_prompt`), with
-      `reasoning=True` when the client prefills `<think>`, and `reasoning=False` for `gameworld`.
-- [ ] The parse rate is high. `Decision.action.parsed` is `False` for unparseable replies, and
-      `Action.truncated` is `True` when `<think>` was never closed. Many of either usually means a
-      wrong template, a stripped special token, or a too-small `max_tokens`.
-- [ ] Executed actions actually move the game. Each chunk runs in wall-clock time (6 × 33 ms), so
-      a slow or shared renderer makes held keys barely move, with no error message. For
-      OmniGameArena, run **one UE5 instance per GPU**.
-- [ ] Episodes are real. In OmniGameArena, start a fresh UE5 process for every `solo_craft`
-      episode; a reused instance ends episodes 2 and 3 immediately and repeats the first score.
-
-**Scoring.** On OmniGameArena, `solo_craft` is scored as `min(V/50, 1)` from the raw points `V`;
-the overall score is the mean over the seven games. ViZDoom counts kills on the battle maps and
-kills minus deaths on the defend maps.
-
-## 7. Adding a new game
-
-1. Fill in `prompts/system_prompt_template.txt` (§2), or call `build_system_prompt` with the game
-   name, rules and `key_bindings` in the training vocabulary. Add `mouse_aim=True` if the mouse
-   controls the camera.
-2. Capture one frame every 200 ms (or at every decision under a paused clock) and call
-   `policy.act(frame)`.
-3. Execute the returned `Action` as in §5: held keys for 6 steps of 33 ms, plus mouse movement and
-   scroll.
-4. Keep the default client (native frame size, `<think>` prefill, standard prompt). Switch to a
-   preset (§6) only to match a paper table.
-
-## 8. Tests
+`keymaps.TO_PLAYWRIGHT` and `keymaps.TO_UE5` map them to browser and Unreal Engine key names;
+`translate_controls` rewrites browser key names in a text into training tokens.
+
+## 6. Running it in a game
+
+1. Write the system prompt for your game (§2).
+2. Capture the current frame and call `policy.act(frame)` once per decision.
+3. Execute the returned `Action` as in §5: held keys for 6 steps of 33 ms, plus mouse movement
+   and scroll.
+4. Call `policy.reset()` when an episode ends.
+
+Before trusting the results, check:
+
+- **Parse rate.** `Decision.action.parsed` should almost always be `True`. Many failures, or many
+  `Action.truncated`, usually mean a replaced chat template, stripped special tokens, a missing
+  prefill, or a too-small `max_tokens`.
+- **Actions take effect.** Each chunk runs in wall-clock time (6 × 33 ms). If the game renders
+  slowly, for example several game instances sharing one GPU, held keys barely move the
+  character and nothing reports an error. Watch a recording before reading any numbers.
+- **Several episodes.** Single episodes vary a lot at `temperature=1.0`.
+
+## 7. Tests
 
 ```bash
 python tests/test_core.py      # or: python -m pytest tests/

gamescaling/__init__.py

@@ -12,11 +12,18 @@ from .action import (
     parse_action,
     split_reasoning,
 )
-from .policy import Decision, GameScalingPolicy, PolicyConfig, build_request, encode_image, finalize
+from .policy import (
+    NO_THINK_PREFILL,
+    THINK_PREFILL,
+    Decision,
+    GameScalingPolicy,
+    PolicyConfig,
+    build_request,
+    encode_image,
+    finalize,
+)
 from .prompt import (
-    TRAINING_SYSTEM_PROMPT,
     USER_TURN_TEXT,
-    build_gameworld_system_prompt,
     build_system_prompt,
     translate_controls,
 )
@@ -26,7 +33,7 @@ __version__ = "0.1.0"
 __all__ = [
     "ACTION_END", "ACTION_START", "CHUNK_MS", "CHUNK_STEPS", "KEY_VOCAB", "NOOP_ACTION_TEXT",
     "STEP_MS", "Action", "parse_action", "split_reasoning",
-    "Decision", "GameScalingPolicy", "PolicyConfig", "build_request", "encode_image", "finalize",
-    "TRAINING_SYSTEM_PROMPT", "USER_TURN_TEXT", "build_gameworld_system_prompt",
-    "build_system_prompt", "translate_controls",
+    "NO_THINK_PREFILL", "THINK_PREFILL", "Decision", "GameScalingPolicy", "PolicyConfig",
+    "build_request", "encode_image", "finalize",
+    "USER_TURN_TEXT", "build_system_prompt", "translate_controls",
 ]

gamescaling/action.py

@@ -24,7 +24,6 @@ from dataclasses import dataclass, field
 
 ACTION_START = "<|action_start|>"
 ACTION_END = "<|action_end|>"
-ACTION_END_TOKEN_ID = 151670
 THINK_OPEN = "<think>"
 THINK_CLOSE = "</think>"
 
@@ -159,7 +158,7 @@ def _parse_strict(tail: str) -> Action:
 
 
 def _parse_lenient(text: str) -> Action:
-    """GameWorld's recovery for malformed replies (see `parse_action`)."""
+    """Recovery for malformed replies (see `parse_action`)."""
     text = text.replace(THINK_OPEN, " ").replace(THINK_CLOSE, " ")
     m = _WRAPPED_INT_RE.search(text)
     if m:
@@ -197,10 +196,10 @@ def _parse_lenient(text: str) -> Action:
 def parse_action(text: str, *, lenient: bool = False) -> Action:
     """Raw model reply -> :class:`Action`. Never raises; failures are no-ops.
 
-    ``lenient=False`` (OmniGameArena / ViZDoom protocol): only a chunk after the
+    ``lenient=False`` (default): only a chunk after the
     last ``</think>`` counts, either tagged or as a bare ``X Y Z ; ...`` body.
 
-    ``lenient=True`` (GameWorld protocol): additionally recover shapes a
+    ``lenient=True``: additionally recover shapes a
     well-trained checkpoint rarely emits -- an action inside an unclosed
     ``<think>``, a 2-number mouse prefix, a keys-only skeleton -- by locating the
     ``;``-separated six-group skeleton anywhere in the reply.

gamescaling/keymaps.py

@@ -10,7 +10,7 @@ from .action import LETTER_KEYS
 
 MOUSE_BUTTONS: dict[str, str] = {"LB": "left", "RB": "right", "MB": "middle"}
 
-# Unreal Engine 5 FKey names (OmniGameArena RemoteInput). Left variants are
+# Unreal Engine 5 FKey names. Left variants are
 # used for the L/R-merged modifier tokens.
 TO_UE5: dict[str, str] = {
     **{c: c for c in LETTER_KEYS},
@@ -28,8 +28,7 @@ TO_UE5: dict[str, str] = {
     "Quote": "Apostrophe",
 }
 
-# Playwright `page.keyboard` names (GameWorld). Letters are lower case because
-# GameWorld's legal-key lists are written that way.
+# Playwright `page.keyboard` names (browser games). Letters are lower case.
 TO_PLAYWRIGHT: dict[str, str] = {
     **{c: c.lower() for c in LETTER_KEYS},
     "zero": "0", "one": "1", "two": "2", "three": "3", "four": "4",

gamescaling/policy.py

@@ -7,7 +7,8 @@ One call to :meth:`GameScalingPolicy.act` is one 200 ms decision:
     ...               then the action the model took (its reasoning is dropped
                       unless `history_think` asks to keep it)
     user              "current game screen" + the current frame
-    assistant         optionally prefilled with "<think>"; the model continues
+    assistant         prefilled with "<think>" (reasoning=True) or
+                      "<think>\\n\\n</think>" (reasoning=False); the model continues
 
 The checkpoint's own chat template renders a past assistant turn without
 reasoning as `<think>\\n\\n</think>\\n\\n<|action_start|>...`, exactly as in
@@ -22,7 +23,7 @@ import logging
 import re
 import time
 from collections import deque
-from dataclasses import dataclass, field, replace
+from dataclasses import dataclass, field
 from typing import Any, NamedTuple
 
 import requests
@@ -30,7 +31,6 @@ from PIL import Image
 
 from .action import (
     ACTION_END,
-    ACTION_END_TOKEN_ID,
     ACTION_START,
     THINK_OPEN,
     Action,
@@ -41,6 +41,11 @@ from .prompt import USER_TURN_TEXT
 
 logger = logging.getLogger(__name__)
 
+# Assistant prefill. Both modes use the same system prompt; the empty think
+# block is exactly how training renders a turn without reasoning.
+THINK_PREFILL = THINK_OPEN
+NO_THINK_PREFILL = "<think>\n\n</think>"
+
 _RESAMPLE = {"bilinear": Image.BILINEAR, "lanczos": Image.LANCZOS, "bicubic": Image.BICUBIC}
 
 
@@ -54,45 +59,24 @@ class PolicyConfig:
     # How many of the most recent past turns keep their <think> text; -1 = all.
     history_think: int = 0
     # Frame sent to the model. None keeps the given game screen as-is.
-    # An exact (w, h), or image_height, is only for a benchmark preset.
     image_size: tuple[int, int] | None = None
-    image_height: int | None = None
     image_format: str = "jpeg"            # jpeg | png
     jpeg_quality: int = 90
     resample: str = "bilinear"
-    # Prefill "<think>" so the model reasons freely and closes it itself. The
-    # system prompt must then be built with reasoning=True.
-    think_prefill: bool = True
+    # True: prefill "<think>" and let the model decide whether and how long to
+    # think. False: prefill an empty think block so the model acts directly.
+    reasoning: bool = True
     temperature: float = 1.0
     max_tokens: int = 2048
     stop: list[str] = field(default_factory=lambda: [ACTION_END])
-    stop_token_ids: list[int] = field(default_factory=list)
-    include_stop_str_in_output: bool = False
+    # Also recover malformed replies (an action inside an unclosed <think>, a
+    # keys-only skeleton). Off by default.
     lenient_parse: bool = False
     timeout: float = 120.0
 
-    @classmethod
-    def preset(cls, benchmark: str, **overrides: Any) -> "PolicyConfig":
-        """Paper-benchmark clients. The default constructor is not one of these.
-
-        Default ``PolicyConfig()`` prefills ``<think>`` and sends the given game
-        frame unchanged. Use a preset only when reproducing a paper table.
-        """
-        presets = {
-            "omnigamearena": cls(image_size=(854, 480)),
-            # Free-form reasoning without a stop string: a model that quotes the
-            # format inside <think> would otherwise be cut off mid-thought.
-            "vizdoom": cls(image_size=(854, 480), stop=[]),
-            "gameworld": cls(
-                image_size=None, image_height=480, image_format="png", resample="lanczos",
-                think_prefill=False, max_tokens=4096, stop=[ACTION_END],
-                stop_token_ids=[ACTION_END_TOKEN_ID], include_stop_str_in_output=True,
-                lenient_parse=True,
-            ),
-        }
-        if benchmark not in presets:
-            raise ValueError(f"unknown benchmark {benchmark!r}; known: {sorted(presets)}")
-        return replace(presets[benchmark], **overrides)
+    @property
+    def prefill(self) -> str:
+        return THINK_PREFILL if self.reasoning else NO_THINK_PREFILL
 
 
 class Turn(NamedTuple):
@@ -119,9 +103,6 @@ def encode_image(image: Image.Image, cfg: PolicyConfig) -> str:
     img = image
     if cfg.image_size is not None and img.size != tuple(cfg.image_size):
         img = img.resize(tuple(cfg.image_size), _RESAMPLE[cfg.resample])
-    elif cfg.image_height and img.height != cfg.image_height:
-        width = max(1, round(img.width * cfg.image_height / img.height))
-        img = img.resize((width, cfg.image_height), _RESAMPLE[cfg.resample])
     buf = io.BytesIO()
     if cfg.image_format == "png":
         img.convert("RGB").save(buf, format="PNG")
@@ -157,14 +138,9 @@ def build_request(messages: list[dict], cfg: PolicyConfig) -> dict:
     }
     if cfg.stop:
         body["stop"] = list(cfg.stop)
-    if cfg.stop_token_ids:
-        body["stop_token_ids"] = list(cfg.stop_token_ids)
-    if cfg.include_stop_str_in_output:
-        body["include_stop_str_in_output"] = True
-    if cfg.think_prefill:
-        body["messages"].append({"role": "assistant", "content": THINK_OPEN})
-        body["continue_final_message"] = True
-        body["add_generation_prompt"] = False
+    body["messages"].append({"role": "assistant", "content": cfg.prefill})
+    body["continue_final_message"] = True
+    body["add_generation_prompt"] = False
     return body
 
 
@@ -175,8 +151,8 @@ def finalize(text: str, finish_reason: str | None, cfg: PolicyConfig) -> str:
     from it. Both are restored so the reply parses the same way as training data.
     """
     text = text or ""
-    if cfg.think_prefill and not text.lstrip().startswith(THINK_OPEN):
-        text = THINK_OPEN + text
+    if not text.lstrip().startswith(THINK_OPEN):
+        text = cfg.prefill + text
     _, tail, closed = split_reasoning(text)
     if closed and ACTION_START not in tail:
         bare = tail.strip()

gamescaling/prompt.py

@@ -1,18 +1,16 @@
-"""System prompts for GameScaling checkpoints.
+"""System prompt for GameScaling checkpoints.
 
-All three benchmarks in the paper use the same skeleton, in this order:
+One skeleton, in this order:
 
 1. opening: gaming expert, the game's name, the 200 ms / 6 x 33 ms contract;
 2. ``**Game Rules**``: what the game is and how it is scored;
-3. controls: the keys this game uses, written in the training vocabulary;
-4. ``**Task Instruction**`` (GameWorld only: one game has several tasks);
-5. ``**Output Format**``: plain, or the empty-think / think+action variant when
-   the client lets the model reason;
-6. ``**Explanation**`` + ``**Key Naming Rules**``.
-
-Only the wording of the controls section and the separators differ between the
-two layouts below. Copy the strings, do not paraphrase them: the checkpoints key
-their output tag off the exact Output Format section.
+3. ``**Common Keys in <game>**``: the keys this game uses, in the training vocabulary;
+4. ``**Output Format**``: empty-think action, or think + action;
+5. ``**Explanation**`` + ``**Key Naming Rules**``.
+
+The ``**...**`` markers are literal plain-text section headers that the model
+saw in training, not Markdown. Copy the strings, do not paraphrase them: the
+checkpoints key their output tags off the exact Output Format section.
 """
 
 from __future__ import annotations
@@ -34,10 +32,7 @@ OPENING = (
     "analysis is no longer valid, or a new objective appears."
 )
 
-OUTPUT_FORMAT_PLAIN = (
-    "<|action_start|>X Y Z ; k1 k2 k3 ; k4 k5 ; k6 ; k7 ; k8 ; k9 k10<|action_end|>"
-)
-OUTPUT_FORMAT_THINK = (
+OUTPUT_FORMAT = (
     "**Action only (usual)**\n"
     "<think></think><|action_start|>X Y Z ; k1 k2 k3 ; k4 k5 ; k6 ; k7 ; k8 ; k9 k10"
     "<|action_end|>\n"
@@ -71,47 +66,23 @@ TAIL = (
 MOUSE_AIM_LINE = "- Mouse X Y -> aim / rotate the camera (X>0 right, Y>0 down)"
 _MOUSE_BUTTON_ALIASES = {"LMB": "LB", "RMB": "RB", "MMB": "MB"}
 
-# The generic prompt of the pre-training mixture: no game information at all,
-# action-only Output Format. Kept for ablations; it does not match a <think>
-# prefill. Inference uses `build_system_prompt`.
-TRAINING_SYSTEM_PROMPT = '''You are an experienced PC game player, proficient in keyboard and mouse operations. Based on the current game screen and the corresponding historical trajectory, plan the next 200ms of actions, consisting of 6 steps. Each step is spaced 33ms apart. Every step lasts 33ms from its start time until the next step begins.
-**Output Format**
-<|action_start|>X Y Z ; k1 k2 k3 ; k4 k5 ; k6 ; k7 ; k8 ; k9 k10<|action_end|>
-**Explanation**
-1. **Mouse Movement**: First, specify the relative displacement X, Y (X>0 means move right, Y>0
-means move down) and scroll amount Z (Z>0 means scroll up).
-2. **Key Sequence**: Then list 6 groups of keys; within each group, keys are separated by spaces,
-and groups are separated by semicolons.
-- Each group can contain up to 4 keys.
-- If a group has no keys, leave it empty but keep the `;`.
-3. Only output a plain string that conforms to the above format — no line breaks and no quotation marks.
-**Key Naming Rules**
-- Number keys `1-9`: use lowercase English words, e.g., `one` represents the `1` key on the keyboard.
-- Function keys `F1-F12`: use capitalized English words, e.g., `One` represents `F1`, `Two` represents `F2`, and so on.
-- Other keys (letters, Shift, Tab, Space, etc.): use the real keyboard name with an initial capital letter, e.g., `A`, `D`, `Shift`, `Space`, etc.'''
-
 USER_TURN_TEXT = "current game screen"
 
 
-def _output_format(reasoning: bool) -> str:
-    return OUTPUT_FORMAT_THINK if reasoning else OUTPUT_FORMAT_PLAIN
-
-
 def build_system_prompt(
     game_name: str,
     *,
     rules: str | None = None,
     key_bindings: Iterable[tuple[str, str]] | dict[str, str] | None = None,
     mouse_aim: bool = True,
-    reasoning: bool = True,
 ) -> str:
-    """Standard key-list layout for any game (ViZDoom, OmniGameArena, or a new game).
+    """The standard system prompt for any game.
 
     ``key_bindings`` maps training tokens to what they do, e.g.
     ``{"W": "Move forward", "LB": "Fire"}``; ``LMB``/``RMB``/``MMB`` are accepted
     as aliases of ``LB``/``RB``/``MB``. ``mouse_aim`` adds the camera line for
-    games that are steered with the mouse. ``reasoning`` must match whether the
-    client prefills ``<think>`` (see ``PolicyConfig.think_prefill``).
+    games that are steered with the mouse. The same prompt serves both
+    ``PolicyConfig(reasoning=True)`` and ``reasoning=False``.
     """
     parts = [OPENING.format(name=game_name)]
     if rules and rules.strip():
@@ -122,36 +93,11 @@ def build_system_prompt(
         common.append(MOUSE_AIM_LINE)
     if common:
         parts.append(f"**Common Keys in {game_name}**\n" + "\n".join(common))
-    parts.append("**Output Format**\n" + _output_format(reasoning))
+    parts.append("**Output Format**\n" + OUTPUT_FORMAT)
     parts.append(TAIL)
     return "\n".join(parts)
 
 
-def build_gameworld_system_prompt(
-    game_name: str | None,
-    *,
-    rules: str | None = None,
-    controls: str | None = None,
-    task: str | None = None,
-    reasoning: bool = False,
-) -> str:
-    """Free-text layout (GameWorld): rules, role/controls and task as paragraphs.
-
-    ``controls`` must already use training key names; see
-    :func:`translate_controls`.
-    """
-    opening = OPENING if game_name else OPENING.replace(", and you are currently playing {name}", "")
-    out = opening.format(name=game_name) + "\n"
-    if rules:
-        out += "\n**Game Rules**\n" + rules + "\n"
-    if controls:
-        out += "\n**Role and Controls**\n" + controls + "\n"
-    if task:
-        out += "\n**Task Instruction**\n" + task + "\n"
-    out += "\n**Output Format**\n" + _output_format(reasoning) + "\n" + TAIL
-    return out.strip()
-
-
 _PW_TO_TOKEN = PLAYWRIGHT_TO_TOKEN
 _PW_KEY_RE = re.compile(r"\b(" + "|".join(map(re.escape, _PW_TO_TOKEN)) + r")\b")
 _DIGIT_WORD = {"0": "zero", "1": "one", "2": "two", "3": "three", "4": "four",
@@ -165,7 +111,7 @@ _LIST_DIGIT_RE = re.compile(r"(?m)^(\s*-\s*)(\d)(?=\s*[:/])")
 
 
 def translate_controls(text: str | None) -> str:
-    """Rewrite a controls paragraph from browser key names into training tokens.
+    """Rewrite key names written for a browser / keyboard API into training tokens.
 
     ``ArrowLeft`` -> ``Left``, ``Escape`` -> ``Esc``, ...; single letters become
     upper case and digits become words, but only where they stand as keys

prompts/example_doom_battle1.txt

@@ -1,12 +1,31 @@
 You are a gaming expert, and you are currently playing Doom Battle-1. You are proficient in keyboard and mouse operations. You understand game mechanics and combat pacing, can quickly extract key information from the current screen, and can think and make precise decisions at critical moments. Based on the current screen, plan the next 200ms of actions, consisting of 6 steps spaced 33ms apart. Each step lasts 33ms, until the next step begins. If the current situation continues the previous strategy, output the action directly. Think only when the situation changes significantly, the previous analysis is no longer valid, or a new objective appears.
 **Game Rules**
-Kill as many monsters as possible; collect health and ammo.
+You are playing Doom, a first-person shooter, in an arena full of monsters. Survive and kill as many monsters as you can.
+
+## What you see on screen
+- First-person view.
+- The crosshair at screen centre marks where shots land.
+- The status bar along the bottom shows HEALTH on the left and AMMO on the right.
+- Monsters keep arriving for the whole episode; some throw fireballs from range.
+
+## Rules
+- Score is the number of monsters you kill in the episode.
+- Firing costs ammo, and the episode ends as soon as your health reaches 0, so dodging matters as much as shooting: standing still in the open is how most episodes end early.
+- Health packs and ammo boxes lie around the map; walk over one to pick it up. Running out of either is fatal, so collect them while fighting.
+- A target is hit when it is under the crosshair, so turn with the mouse until the monster is centred before firing.
+- The map is one large open arena, so monsters can close in from any direction.
 **Common Keys in Doom Battle-1**
 - W -> Move forward
-- S -> Move backward
 - A -> Strafe left
+- S -> Move backward
 - D -> Strafe right
-- LB -> Fire
+- LB -> Fire weapon
+- Ctrl -> Fire weapon (keyboard alternative)
+- Shift -> Run
+- Space -> Open doors / use switches
+- one -> Select weapon slot 1
+- two -> Select weapon slot 2
+- three -> Select weapon slot 3
 - Mouse X Y -> aim / rotate the camera (X>0 right, Y>0 down)
 **Output Format**
 **Action only (usual)**

requirements.txt

@@ -2,7 +2,5 @@
 requests>=2.31
 Pillow>=10.0
 
-# ViZDoom suite (integrations/vizdoom):
-#   pip install vizdoom==1.2.4
 # Serving the checkpoint (separate environment is fine):
 #   pip install "vllm>=0.17"      # Qwen3.5 architecture; tested with 0.17.0

tests/test_core.py

@@ -10,7 +10,6 @@ from PIL import Image  # noqa: E402
 from gamescaling import (  # noqa: E402
     NOOP_ACTION_TEXT,
     PolicyConfig,
-    build_gameworld_system_prompt,
     build_request,
     build_system_prompt,
     finalize,
@@ -61,29 +60,30 @@ def test_lenient_recovers_skeleton():
 
 
 def test_finalize_restores_prefill_and_stop():
-    cfg = PolicyConfig()
-    out = finalize("\n\n</think>\n\n<|action_start|>0 0 0 ; W ; ; ; ; ; ", "stop", cfg)
+    out = finalize("\n\n</think>\n\n<|action_start|>0 0 0 ; W ; ; ; ; ; ", "stop", PolicyConfig())
     assert out.startswith("<think>") and out.endswith("<|action_end|>")
     assert parse_action(out).steps[0] == ["W"]
+    out = finalize("\n\n<|action_start|>0 0 0 ; ; D ; ; ; ; ", "stop", PolicyConfig(reasoning=False))
+    assert out.startswith("<think>\n\n</think>\n\n<|action_start|>")
+    assert parse_action(out).steps[1] == ["D"]
 
 
 def test_request_shape():
     cfg = PolicyConfig(model="m")
-    assert cfg.image_size is None
+    assert cfg.image_size is None and cfg.reasoning
     body = build_request([{"role": "system", "content": "s"}], cfg)
     assert body["messages"][-1] == {"role": "assistant", "content": "<think>"}
     assert body["continue_final_message"] and not body["add_generation_prompt"]
     assert body["stop"] == ["<|action_end|>"] and body["skip_special_tokens"] is False
-    oga = PolicyConfig.preset("omnigamearena")
-    assert oga.image_size == (854, 480) and oga.think_prefill
-    gw = build_request([], PolicyConfig.preset("gameworld"))
-    assert "continue_final_message" not in gw and gw["stop_token_ids"] == [151670]
+    body = build_request([{"role": "system", "content": "s"}], PolicyConfig(reasoning=False))
+    assert body["messages"][-1] == {"role": "assistant", "content": "<think>\n\n</think>"}
+    assert body["continue_final_message"]
 
 
 def test_image_encoding():
     img = Image.new("RGB", (1280, 720))
     assert encode_image(img, PolicyConfig()).startswith("data:image/jpeg;base64,")
-    assert encode_image(img, PolicyConfig.preset("gameworld")).startswith("data:image/png;base64,")
+    assert encode_image(img, PolicyConfig(image_format="png")).startswith("data:image/png;base64,")
 
 
 def test_prompt_sections_in_order():
@@ -92,10 +92,6 @@ def test_prompt_sections_in_order():
              "**Action only (usual)**", "**Explanation**", "**Key Naming Rules**"]
     idx = [p.index(s) for s in order]
     assert idx == sorted(idx)
-    gw = build_gameworld_system_prompt("2048", rules="Merge tiles.", controls="Use the arrow keys.",
-                                       task="Reach 256.")
-    assert "\n\n**Game Rules**\nMerge tiles.\n\n**Role and Controls**\n" in gw
-    assert "**Action only" not in gw
 
 
 def test_prompt_template_file_matches_builder():
@@ -109,6 +105,16 @@ def test_prompt_template_file_matches_builder():
     assert filled == build_system_prompt("Doom", rules="Kill monsters.", key_bindings=keys)
 
 
+def test_doom_example_matches_builder():
+    root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+    with open(os.path.join(root, "prompts", "example_doom_battle1.txt")) as f:
+        example = f.read().rstrip("\n")
+    rules = example.split("**Game Rules**\n", 1)[1].split("\n**Common Keys in", 1)[0]
+    keys = example.split("**Common Keys in Doom Battle-1**\n", 1)[1].split("\n**Output Format**", 1)[0]
+    bindings = [line[2:].split(" -> ", 1) for line in keys.splitlines() if not line.startswith("- Mouse X Y")]
+    assert example == build_system_prompt("Doom Battle-1", rules=rules, key_bindings=bindings)
+
+
 def test_translate_controls():
     text = "- ArrowLeft: move left\n- a/d: strafe\n- 1-9: pick item\nPress Escape to pause."
     out = translate_controls(text)