Model card: plain title, compact authors, file tree, deployment and evaluation focus

README.md

@@ -21,26 +21,16 @@ tags:
 
 <div align="center">
 
-<h1 align="center" style="font-size:3.4em;line-height:1.1;margin:0.2em 0 0.15em;">Scaling in Games</h1>
+# Scaling in Games: Continued Pre-Training for Embodied Agents in Diverse Virtual Worlds
 
-<h2 align="center" style="font-size:1.6em;line-height:1.3;font-weight:600;margin:0 0 0.9em;border:none;">Continued Pre-Training for Embodied Agents in Diverse Virtual Worlds</h2>
-
-<p align="center" style="font-size:1.08em;line-height:1.8;">
-<b>Kuan Zhang</b><sup>1,2,*,‡</sup> &nbsp; <b>Yukun Chen</b><sup>1,3,*,‡</sup> &nbsp; <b>Zhihao Yang</b><sup>1,3,*,‡</sup> &nbsp; Yue Su<sup>4</sup><br>
-Xiangnan Wu<sup>3</sup> &nbsp; Zirong Chen<sup>2</sup> &nbsp; Run Luo<sup>1,5,‡</sup> &nbsp; Jinkun Hou<sup>6</sup><br>
-Tao Tan<sup>1</sup> &nbsp; <b>Yinhe Zheng</b><sup>1,†</sup> &nbsp; <b>Yiming Li</b><sup>2,†</sup>
-</p>
-
-<p align="center" style="font-size:0.9em;line-height:1.75;white-space:nowrap;">
-<sup>1</sup>miHoYo Honkai AI R&amp;D Team &nbsp;·&nbsp; <sup>2</sup>College AI, Tsinghua University<br>
-<sup>3</sup>University of Chinese Academy of Sciences &nbsp;·&nbsp; <sup>4</sup>MMLab, The University of Hong Kong<br>
-<sup>5</sup>National University of Singapore &nbsp;·&nbsp; <sup>6</sup>Peking University
+<p align="center" style="line-height:1.5;margin:0.4em 0 0.3em;">
+<b>Kuan Zhang</b><sup>1,2,*,‡</sup>, <b>Yukun Chen</b><sup>1,3,*,‡</sup>, <b>Zhihao Yang</b><sup>1,3,*,‡</sup>, Yue Su<sup>4</sup>, Xiangnan Wu<sup>3</sup>, Zirong Chen<sup>2</sup>, Run Luo<sup>1,5,‡</sup>, Jinkun Hou<sup>6</sup>, Tao Tan<sup>1</sup>, <b>Yinhe Zheng</b><sup>1,†</sup>, <b>Yiming Li</b><sup>2,†</sup>
 </p>
 
-<p align="center" style="font-size:0.9em;">
-<sup>*</sup>Equal contribution &nbsp;&nbsp; <sup>†</sup>Corresponding author &nbsp;&nbsp; <sup>‡</sup>Work done while interning at miHoYo
+<p align="center" style="font-size:0.85em;line-height:1.45;margin:0 0 0.6em;">
+<sup>1</sup>miHoYo Honkai AI R&amp;D Team &nbsp; <sup>2</sup>College AI, Tsinghua University &nbsp; <sup>3</sup>University of Chinese Academy of Sciences &nbsp; <sup>4</sup>MMLab, The University of Hong Kong &nbsp; <sup>5</sup>National University of Singapore &nbsp; <sup>6</sup>Peking University<br>
+<sup>*</sup>Equal contribution &nbsp; <sup>†</sup>Corresponding author &nbsp; <sup>‡</sup>Work done while interning at miHoYo
 </p>
-
 <p align="center">
   <img src="https://huggingface.co/THUSI-Lab/GameScaling/resolve/main/assets/logos/affiliations.png" alt="miHoYo · Tsinghua University · University of Chinese Academy of Sciences · The University of Hong Kong" width="100%" style="width:100%;max-width:100%;">
 </p>
@@ -60,99 +50,75 @@ Tao Tan<sup>1</sup> &nbsp; <b>Yinhe Zheng</b><sup>1,†</sup> &nbsp; <b>Yiming L
   <img src="https://huggingface.co/THUSI-Lab/GameScaling/resolve/main/assets/teaser.jpg" alt="From one physical world to diverse virtual worlds" width="100%">
 </p>
 
-<p align="center"><i>Embodied learning in the physical world benefits from shared physics, visuals and controls. Virtual worlds break that assumption: every game brings its own. Does scaling still carry over?</i></p>
-
 ## Overview
 
-GameScaling models are Qwen3.5 vision-language models with continued pre-training (CPT) on
-keyboard-and-mouse gameplay. Each call reads the current game frame (plus a short history of past
-frames and actions) and outputs the next **200 ms of keyboard and mouse actions**: 6 steps of
-33 ms each, plus one mouse movement and scroll for the whole chunk. The model can optionally
-think first.
-
-The paper is a controlled study of CPT along three axes: data volume, data diversity and model
-size. It covers 120 training configurations: four Qwen3.5 backbones (0.8B–27B), six gameplay
-budgets (50–1,600 h) and five data mixtures (S1–S5) from an 80-game, mostly 3D corpus. Evaluation
-moves from worlds in the training corpus (MCU / Minecraft, ViZDoom) to unseen related 3D worlds
-(OmniGameArena) and unseen distant, mostly 2D worlds (GameWorld).
-
-This repository releases eight checkpoints (four model sizes × two mixtures, all at 1,600 h), the
-inference client, and the standard system-prompt template.
-
-**Key findings**
-
-- **Data, diversity and size help different worlds.** More data improves play in worlds from the
-  training corpus. Diversity improves transfer to related unseen worlds, with limited gains in
-  distant ones.
-- **Small and fast wins in real time.** GameScaling takes 0.13–0.69 s per step, versus 4–24 s for
-  frontier models, and scores highest when the world does not pause.
-- **More steps help, but only after pre-training.** Raising the MCU step cap from 600 to 3,000
-  lifts GameScaling-9B-S1 from 50.9% to 69.3%; the untrained Qwen3.5-9B stays at 4.0%.
-
-## Results
-
-Average time-paused score (%) of the released 1,600 h checkpoints, as reported on the
-[project page](https://gamescaling.github.io).
-
-| Size | Model | MCU | ViZDoom | OmniGameArena | GameWorld |
-|---|---|:---:|:---:|:---:|:---:|
-| 0.8B | Qwen3.5 (no CPT) | 0.0 | 0.9 | 1.7 | 13.8 |
-| | GameScaling-S1 | **44.0** | 5.4 | 7.1 | 14.4 |
-| | GameScaling-S3 | 36.0 | **11.9** | **10.2** | **20.3** |
-| 2B | Qwen3.5 (no CPT) | 0.0 | 0.9 | 1.8 | **19.8** |
-| | GameScaling-S1 | **49.3** | 10.7 | 5.9 | 15.1 |
-| | GameScaling-S3 | 48.0 | **11.2** | **13.9** | 19.5 |
-| 9B | Qwen3.5 (no CPT) | 3.3 | 2.8 | 5.2 | **29.2** |
-| | GameScaling-S1 | **54.7** | 8.1 | 6.0 | 17.0 |
-| | GameScaling-S3 | 44.7 | **15.9** | **27.3** | 26.2 |
-| 27B | Qwen3.5 (no CPT) | 9.3 | 5.6 | 9.7 | **29.3** |
-| | GameScaling-S1 | **63.3** | 5.3 | 7.9 | 12.5 |
-| | GameScaling-S3 | 54.0 | **14.7** | **25.7** | 25.3 |
-
-S1 is the specialist: it is best on Minecraft (MCU). S3 trades some Minecraft score for clear
-gains in unseen 3D worlds (OmniGameArena). Neither closes the gap in distant 2D worlds (GameWorld),
-which suggests that CPT alone is not enough for a generalist agent. Base and S3 numbers come from
-the paper's tables; S1 numbers are read from the 1,600 h column of the paper's Figure 4.
+GameScaling models are Qwen3.5 vision-language models with continued pre-training on
+keyboard-and-mouse gameplay. Each call takes the current game frame (plus a short history of past
+frames and actions) and returns the next **200 ms of keyboard and mouse actions**: 6 steps of
+33 ms each, plus one mouse movement and scroll for the whole chunk. The model may think before it
+acts.
+
+This repository contains eight checkpoints, the inference client, and the standard system-prompt
+template. It is everything needed to serve a checkpoint and run it in a game; it contains no
+training code. Paper results and analysis are on the [project page](https://gamescaling.github.io).
 
 ## Checkpoints
 
-| Model | Folder | Mixture |
-|---|---|---|
-| GameScaling-0.8B-S1 | `models/0p8b_s1_1600h_gs6736` | S1 |
-| GameScaling-0.8B-S3 | `models/0p8b_s3_1600h_gs6760` | S3 |
-| GameScaling-2B-S1 | `models/2b_s1_1600h_gs6736` | S1 |
-| GameScaling-2B-S3 | `models/2b_s3_1600h_gs6760` | S3 |
-| GameScaling-9B-S1 | `models/9b_s1_1600h_gs6736` | S1 |
-| GameScaling-9B-S3 | `models/9b_s3_1600h_gs6760` | S3 |
-| GameScaling-27B-S1 | `models/27b_s1_1600h_gs6995` | S1 |
-| GameScaling-27B-S3 | `models/27b_s3_1600h_gs7020` | S3 |
-
-- **S1**: trained on Minecraft gameplay only.
-- **S3**: a balanced mixture. Minecraft is cut to 27%, and the rest comes from other games in an
-  80-game, mostly 3D corpus.
-- Folder names: `<size>_<mixture>_<hours of gameplay>_gs<training steps>`.
-- All weights are bfloat16. Every folder has its own `chat_template.jinja`; keep it (see below).
-
-In the paper, more data helps most in worlds that are in the training corpus. More diversity (S3)
-helps most in unseen 3D worlds whose actions resemble training. See the
-[project page](https://gamescaling.github.io) for the full results.
-
-## Repository layout
+| Model | Folder | Weights | GPUs (bf16) |
+|---|---|---:|:---:|
+| GameScaling-0.8B-S1 | `models/0p8b_s1_1600h_gs6736` | 2.2 GB | 1 |
+| GameScaling-0.8B-S3 | `models/0p8b_s3_1600h_gs6760` | 2.2 GB | 1 |
+| GameScaling-2B-S1 | `models/2b_s1_1600h_gs6736` | 5.4 GB | 1 |
+| GameScaling-2B-S3 | `models/2b_s3_1600h_gs6760` | 5.4 GB | 1 |
+| GameScaling-9B-S1 | `models/9b_s1_1600h_gs6736` | 18.8 GB | 1 |
+| GameScaling-9B-S3 | `models/9b_s3_1600h_gs6760` | 18.8 GB | 1 |
+| GameScaling-27B-S1 | `models/27b_s1_1600h_gs6995` | 54.7 GB | 2 (tensor parallel) |
+| GameScaling-27B-S3 | `models/27b_s3_1600h_gs7020` | 54.7 GB | 2 (tensor parallel) |
 
-```
-models/                      eight checkpoints (Qwen3.5 architecture)
-prompts/
-  system_prompt_template.txt standard system prompt; fill in three placeholders for any game
-  example_doom_battle1.txt   a filled-in example
-gamescaling/                 inference client; depends only on requests + Pillow
-  prompt.py                  system prompts
-  policy.py                  multi-turn client: history window, image encoding, request body
-  action.py                  action format and parser (200 ms chunk -> 6 x 33 ms key/mouse)
-  keymaps.py                 training key vocabulary <-> browser / Playwright key names
-scripts/serve_vllm.sh        OpenAI-compatible server
-tests/test_core.py           offline unit tests (no GPU or server needed)
-assets/                      teaser and institution logos used in this card
-LICENSE                      Apache License 2.0
+- **S1** is trained on Minecraft gameplay only. **S3** is a balanced mixture: 27% Minecraft, the
+  rest from other games. All checkpoints use 1,600 h of gameplay.
+- Folder names are `<size>_<mixture>_<hours>h_gs<training steps>`. Weights are bfloat16, from the
+  Qwen3.5-0.8B / 2B / 9B / 27B backbones.
+
+## File tree
+
+```text
+THUSI-Lab/GameScaling
+├── models/
+│   ├── 0p8b_s1_1600h_gs6736/              one folder per checkpoint (eight in total)
+│   │   ├── config.json                    Qwen3_5ForConditionalGeneration, bfloat16
+│   │   ├── model.safetensors              weights (27B: model-0000{1,2}-of-00002 + index.json)
+│   │   ├── chat_template.jinja            REQUIRED: renders history turns exactly as in training
+│   │   ├── tokenizer.json
+│   │   ├── tokenizer_config.json
+│   │   ├── generation_config.json
+│   │   ├── preprocessor_config.json       image processor
+│   │   ├── processor_config.json
+│   │   └── video_preprocessor_config.json
+│   ├── 0p8b_s3_1600h_gs6760/
+│   ├── 2b_s1_1600h_gs6736/
+│   ├── 2b_s3_1600h_gs6760/
+│   ├── 9b_s1_1600h_gs6736/
+│   ├── 9b_s3_1600h_gs6760/
+│   ├── 27b_s1_1600h_gs6995/
+│   └── 27b_s3_1600h_gs7020/
+├── prompts/
+│   ├── system_prompt_template.txt         standard system prompt with 3 placeholders (§2)
+│   └── example_doom_battle1.txt           a filled-in example
+├── gamescaling/                           inference client (requests + Pillow only)
+│   ├── __init__.py
+│   ├── prompt.py                          build_system_prompt / build_gameworld_system_prompt
+│   ├── policy.py                          GameScalingPolicy, PolicyConfig, presets, request body
+│   ├── action.py                          action format, parser, 74-token key vocabulary
+│   └── keymaps.py                         training key names <-> browser / Playwright keys
+├── scripts/
+│   └── serve_vllm.sh                      start an OpenAI-compatible vLLM server
+├── tests/
+│   └── test_core.py                       offline tests: prompts, requests, parsing (no GPU)
+├── assets/                                images used in this card
+├── requirements.txt                       client dependencies
+├── LICENSE                                Apache License 2.0
+└── README.md
 ```
 
 ## 1. Download and serve
@@ -163,6 +129,7 @@ Download only the checkpoint you need:
 pip install -U huggingface_hub
 huggingface-cli download THUSI-Lab/GameScaling \
     --include "models/9b_s1_1600h_gs6736/*" "gamescaling/*" "prompts/*" "scripts/*" \
+              "tests/*" "requirements.txt" \
     --local-dir ./GameScaling
 cd GameScaling
 ```
@@ -175,10 +142,28 @@ bash scripts/serve_vllm.sh models/9b_s1_1600h_gs6736 8000        # 0.8B-9B fit o
 bash scripts/serve_vllm.sh models/27b_s1_1600h_gs6995 8000 2      # 27B: tensor parallel 2
 ```
 
-The script serves the model as `gamescaling`. The client connects to `http://127.0.0.1:8000/v1`
-by default. Use the `chat_template.jinja` in the checkpoint folder (vLLM loads it by default). **Do
-not** replace it with the stock Qwen template: this one renders past turns without reasoning as
-`<think>\n\n</think>\n\n<|action_start|>...`, which is token-for-token what training used.
+`serve_vllm.sh CKPT [PORT] [TP]` starts vLLM with `--served-model-name gamescaling`, bfloat16,
+`--max-model-len 32768` (system prompt + 5 history frames + reasoning budget) and
+`--limit-mm-per-prompt '{"image": 8}'` (5 history frames + the current one, with headroom). Set
+`GPU_MEM_UTIL` to change the memory fraction (default 0.85).
+
+Check that the server is up and the client can reach it:
+
+```bash
+curl -s http://127.0.0.1:8000/v1/models          # should list "gamescaling"
+pip install -r requirements.txt
+python tests/test_core.py                         # offline checks, no server needed
+```
+
+Things to keep as shipped:
+
+- **Chat template.** Use the `chat_template.jinja` in the checkpoint folder (vLLM loads it by
+  default). Do not replace it with the stock Qwen template: this one renders past turns without
+  reasoning as `<think>\n\n</think>\n\n<|action_start|>...`, token-for-token as in training.
+- **Special tokens.** Requests must set `skip_special_tokens: false`, or the server strips the
+  `<|action_start|>` / `<|action_end|>` tags. `build_request` already does this.
+- **One server per GPU group.** One vLLM replica can serve several games at once, but long
+  prefills (one image per history turn) make each step slower when many games share it.
 
 ## 2. System prompt template
 
@@ -334,14 +319,16 @@ Training key vocabulary (74 tokens, case-sensitive, `gamescaling.action.KEY_VOCA
 Tokens outside the vocabulary are dropped and recorded in `Action.dropped`.
 `keymaps.TO_PLAYWRIGHT` maps them to browser key names.
 
-## 6. Paper evaluation settings
+## 6. Evaluation
 
-The paper evaluates on MCU (Minecraft), ViZDoom, OmniGameArena and GameWorld. The main tables use
-a **time-paused** clock: the game freezes while the model thinks and advances only while a chunk
-executes. The project page also reports a real-time setting, where the game keeps running during
-inference. Scores from different clocks are not comparable.
+The paper evaluates on four benchmarks: MCU (Minecraft), ViZDoom, OmniGameArena and GameWorld.
+To get comparable numbers, keep these settings fixed.
 
-Client presets for reproducing the paper:
+**Clock.** The standard setting is **time-paused**: the game freezes while the model thinks and
+advances only while a chunk executes. In a real-time setting the game keeps running during
+inference, so latency becomes part of the score. Do not compare scores across clocks.
+
+**Client presets.** Each benchmark has a preset in `PolicyConfig.preset(...)`:
 
 | Preset | Frame | `<think>` prefill | Other |
 |---|---|---|---|
@@ -349,13 +336,25 @@ Client presets for reproducing the paper:
 | `vizdoom` | resized to 854×480, JPEG | yes | no stop string; 4 maps × 10 episodes, at most 350 chunks each |
 | `gameworld` | height 480, PNG (Lanczos) | no | `max_tokens=4096`, lenient parsing; use `build_gameworld_system_prompt(..., reasoning=False)` |
 
-All presets keep 5 history turns (frame + action) at temperature 1.0. On OmniGameArena,
-`solo_craft` is scored as `min(V/50, 1)` from the raw points `V`, and the overall score is the
-mean over the seven games.
-
-When running OmniGameArena, use **one UE5 instance per GPU**. Actions are executed in wall-clock
-time (6 × 33 ms), and instances sharing a GPU's renderer barely move. Scores drop sharply with no
-error message.
+All presets keep 5 history turns (frame + action, no reasoning) at temperature 1.0. Run at least
+3 episodes per game; single episodes vary a lot.
+
+**Checklist before reading a score:**
+
+- [ ] The system prompt is the template from §2 (or `build_system_prompt`), with
+      `reasoning=True` when the client prefills `<think>`, and `reasoning=False` for `gameworld`.
+- [ ] The parse rate is high. `Decision.action.parsed` is `False` for unparseable replies, and
+      `Action.truncated` is `True` when `<think>` was never closed. Many of either usually means a
+      wrong template, a stripped special token, or a too-small `max_tokens`.
+- [ ] Executed actions actually move the game. Each chunk runs in wall-clock time (6 × 33 ms), so
+      a slow or shared renderer makes held keys barely move, with no error message. For
+      OmniGameArena, run **one UE5 instance per GPU**.
+- [ ] Episodes are real. In OmniGameArena, start a fresh UE5 process for every `solo_craft`
+      episode; a reused instance ends episodes 2 and 3 immediately and repeats the first score.
+
+**Scoring.** On OmniGameArena, `solo_craft` is scored as `min(V/50, 1)` from the raw points `V`;
+the overall score is the mean over the seven games. ViZDoom counts kills on the battle maps and
+kills minus deaths on the defend maps.
 
 ## 7. Adding a new game