IndusHomie commited on
Commit
1d1237b
·
verified ·
1 Parent(s): 9a21ce7

Plainer copy

Browse files
Files changed (2) hide show
  1. README.md +120 -145
  2. app.py +22 -30
README.md CHANGED
@@ -1,145 +1,120 @@
1
- ---
2
- title: Prime Rush — a battle royale as an MCP environment
3
- emoji: 🎯
4
- colorFrom: blue
5
- colorTo: yellow
6
- sdk: docker
7
- app_port: 7860
8
- pinned: false
9
- license: apache-2.0
10
- short_description: A live battle royale as six MCP tools
11
- ---
12
-
13
- # Prime Rush as an MCP environment
14
-
15
- A live battle-royale match from a shipped commercial game, exposed to an LLM
16
- agent as six MCP tools. An agent is dropped in with a shrinking safe circle;
17
- anything outside it takes damage until it dies.
18
-
19
- Built for the [Archipelago](https://github.com/Mercor-Intelligence/archipelago)
20
- harness. The layout, the structured-output convention and the schema
21
- flattening mirror that repo's own MCP servers, so this can be reviewed next to
22
- the servers already in it.
23
-
24
- ## The thing worth checking first
25
-
26
- **The match is frozen between tool calls.** The simulation advances only
27
- inside a tool that says it advances it, by exactly the amount that tool asks
28
- for. A model may spend thirty seconds reasoning and the game is in precisely
29
- the state it left it. A slow model is not punished and a fast one is not
30
- rewarded; the same seed and the same calls produce the same match.
31
-
32
- Almost no real-time game can be evaluated this way. It is the reason this is a
33
- test of spatial decision-making rather than of latency.
34
-
35
- ## What the agent gets, and what it does not
36
-
37
- | Tool | Advances the match |
38
- |---|---|
39
- | `get_situation` | no |
40
- | `turn_to_bearing` | yes, by the time the turn takes |
41
- | `move` | yes, by the seconds requested |
42
- | `hold_position` | yes |
43
- | `shoot` | yes |
44
- | `finish` | no |
45
-
46
- There is deliberately **no `move_to(x, z)`** and **no "the zone is 40 degrees
47
- to your left" field**. `get_situation` returns raw coordinates: where you are,
48
- which way you are facing, and where the circle's centre is. Turning that into
49
- a heading is the skill being measured. A server that pre-computed the bearing
50
- would score every model identically and measure nothing.
51
-
52
- Movement is body-relative for the same reason — "go north" would let a model
53
- skip reasoning about its own heading entirely.
54
-
55
- ## How grading works
56
-
57
- Archipelago grades by snapshotting the environment and diffing it, so a score
58
- that only ever existed in a tool response is invisible to the grader. Every
59
- call that advances the match rewrites `INDUS_OUTCOME_PATH`
60
- (default `/filesystem/outcome.json`):
61
-
62
- ```json
63
- {
64
- "task": "zone_survival",
65
- "frac_inside_zone": 0.82,
66
- "seconds_scored": 100.0,
67
- "alive": true,
68
- "player_rank": 12,
69
- "tool_calls_that_advanced_time": 14
70
- }
71
- ```
72
-
73
- The score comes from the game's own state, never from anything the agent said,
74
- so an agent cannot talk its way to a pass. Because the file is rewritten after
75
- every move rather than at the end, a run that crashed, timed out or gave up is
76
- still graded on what it actually did.
77
-
78
- `grading_eval/indus_zone_survival.py` is a reference eval definition that
79
- reads this file and thresholds it, with the snippet to register it. The task
80
- pack also ships `file_diff_check` and `output_llm` verifiers, both of which
81
- work in stock Archipelago with no additions.
82
-
83
- ## What this Space runs
84
-
85
- The tools, their schemas, the scoring and the outcome file are the real ones —
86
- this page imports the same `indus_mcp` package a lab would run.
87
-
88
- The **game** is not. A Space has no Unity server, so the tools here drive
89
- `tests/fake_game.py`, a stand-in that speaks the same bridge protocol and
90
- reproduces the real game's response characteristics — including a look input
91
- that turns at 0.075 of the requested angle, so an agent behaves here the way
92
- it behaves there.
93
-
94
- For the real game, the environment is hosted. Get in touch.
95
-
96
- ## Running it yourself
97
-
98
- ```bash
99
- pip install fastmcp pydantic
100
- python tests/test_end_to_end.py # 28 checks, no game build needed
101
- python tests/test_grading_eval.py # 9 checks
102
- ```
103
-
104
- Against a real match:
105
-
106
- ```bash
107
- export INDUS_BRIDGE_HOST=... INDUS_BRIDGE_PORT=...
108
- python -m indus_mcp.main # stdio, the transport Archipelago uses
109
- ```
110
-
111
- Under Archipelago, `POST /apps` with `tasks/zone_survival/mcp_config.json`.
112
-
113
- ### Configuration
114
-
115
- | Variable | Meaning | Default |
116
- |---|---|---|
117
- | `INDUS_BRIDGE_HOST` / `PORT` | the running match to attach to | `127.0.0.1:7850` |
118
- | `INDUS_TASK` | task name | `zone_survival` |
119
- | `INDUS_BUDGET_SECONDS` | game time the agent gets | `100` |
120
- | `INDUS_FF_RADIUS` | hand over once the circle is this small | `350` |
121
- | `INDUS_OUTCOME_PATH` | where the grader reads the result | `/filesystem/outcome.json` |
122
- | `MCP_TRANSPORT` | `stdio` or `http` | `stdio` |
123
-
124
- `INDUS_FF_RADIUS` matters more than it looks. Before the circle has closed it
125
- covers most of the map and every agent is trivially inside it, so a run
126
- started at match open measures nothing.
127
-
128
- ## Known limitations
129
-
130
- - **No vertical aim.** `shoot` fires horizontally. Aim pitch is not yet
131
- carried in the observation, so an agent could not verify the effect of a
132
- `pitch` argument; it arrives once the observation does.
133
- - **No occlusion.** `get_situation` reports nearby enemies whether or not
134
- there is a wall in the way. Fine for the zone task, wrong for combat.
135
- - **One task.** The session layer is task-aware, so a second task is a new
136
- handover condition and a new metric, not a new server.
137
- - **One seat.** A match holds one agent today. Several agents in one match is
138
- the next change.
139
- - **No scores published yet.** Comparative numbers follow once the benchmark
140
- is finalised; a ranking is only worth publishing when every layout is
141
- winnable, and that is pending a game build.
142
-
143
- ---
144
-
145
- Built by SuperGaming, from the live build of Prime Rush. Apache-2.0.
 
1
+ ---
2
+ title: Prime Rush — a battle royale as an MCP environment
3
+ emoji: 🎯
4
+ colorFrom: blue
5
+ colorTo: yellow
6
+ sdk: docker
7
+ app_port: 7860
8
+ pinned: false
9
+ license: apache-2.0
10
+ short_description: A live battle royale as six MCP tools
11
+ ---
12
+
13
+ # Prime Rush as an MCP environment
14
+
15
+ A battle-royale match exposed to an LLM agent as six MCP tools. A safe circle
16
+ shrinks on a timer; anything outside it takes damage until it dies.
17
+
18
+ Built for the [Archipelago](https://github.com/Mercor-Intelligence/archipelago)
19
+ harness, following the layout and conventions of the MCP servers already in
20
+ that repo.
21
+
22
+ ## Timing
23
+
24
+ The match only advances inside a tool call, by the amount that call asks for.
25
+ A model can take as long as it likes to decide and the game waits, so a slow
26
+ model is not penalised. The same seed and the same calls give the same match.
27
+
28
+ ## Tools
29
+
30
+ | Tool | Advances the match |
31
+ |---|---|
32
+ | `get_situation` | no |
33
+ | `turn_to_bearing` | yes, by the time the turn takes |
34
+ | `move` | yes, by the seconds requested |
35
+ | `hold_position` | yes |
36
+ | `shoot` | yes |
37
+ | `finish` | no |
38
+
39
+ `get_situation` returns coordinates: position, heading, the circle's centre
40
+ and radius, nearby enemies, time left. There is no `move_to` and no bearing to
41
+ the circle — working that out is the task. Movement is relative to the
42
+ direction the agent is facing, so it has to turn first.
43
+
44
+ ## Scoring
45
+
46
+ Each call that advances the match writes `INDUS_OUTCOME_PATH`
47
+ (default `/filesystem/outcome.json`) from the game state:
48
+
49
+ ```json
50
+ {
51
+ "task": "zone_survival",
52
+ "frac_inside_zone": 0.82,
53
+ "seconds_scored": 100.0,
54
+ "alive": true,
55
+ "player_rank": 12,
56
+ "tool_calls_that_advanced_time": 14
57
+ }
58
+ ```
59
+
60
+ Archipelago grades by snapshotting the environment, so the score is read from
61
+ that file rather than from the transcript. It is rewritten after every move,
62
+ so a run that crashed or stopped early is still scored on what it did.
63
+
64
+ `grading_eval/indus_zone_survival.py` is a reference eval definition that
65
+ reads the file and thresholds it, with the snippet to register it. The task
66
+ pack also ships `file_diff_check` and `output_llm` verifiers, which work in
67
+ stock Archipelago.
68
+
69
+ ## What this Space runs
70
+
71
+ The tools, their schemas and the scoring are the ones a lab would run. The
72
+ game is not: a Space has no Unity server, so the tools drive
73
+ `tests/fake_game.py`, a stand-in that speaks the same bridge protocol and
74
+ reproduces the real game's response characteristics. The map on the page
75
+ shows only what the tools returned.
76
+
77
+ For the real game, the environment is hosted. Get in touch.
78
+
79
+ ## Running it
80
+
81
+ ```bash
82
+ pip install fastmcp pydantic
83
+ python tests/test_end_to_end.py # 28 checks, no game build needed
84
+ python tests/test_grading_eval.py # 9 checks
85
+ ```
86
+
87
+ Against a real match:
88
+
89
+ ```bash
90
+ export INDUS_BRIDGE_HOST=... INDUS_BRIDGE_PORT=...
91
+ python -m indus_mcp.main # stdio, the transport Archipelago uses
92
+ ```
93
+
94
+ Under Archipelago, `POST /apps` with `tasks/zone_survival/mcp_config.json`.
95
+
96
+ | Variable | Meaning | Default |
97
+ |---|---|---|
98
+ | `INDUS_BRIDGE_HOST` / `PORT` | the running match to attach to | `127.0.0.1:7850` |
99
+ | `INDUS_TASK` | task name | `zone_survival` |
100
+ | `INDUS_BUDGET_SECONDS` | game time the agent gets | `100` |
101
+ | `INDUS_FF_RADIUS` | hand over once the circle is this small | `350` |
102
+ | `INDUS_OUTCOME_PATH` | where the grader reads the result | `/filesystem/outcome.json` |
103
+ | `MCP_TRANSPORT` | `stdio` or `http` | `stdio` |
104
+
105
+ `INDUS_FF_RADIUS` matters: before the circle has closed it covers most of the
106
+ map and any agent is inside it, so a run started at match open measures
107
+ nothing.
108
+
109
+ ## Limitations
110
+
111
+ - `shoot` is horizontal only. Aim pitch is not in the observation yet, so an
112
+ agent could not check the effect of a `pitch` argument.
113
+ - No line of sight: nearby enemies are reported through walls.
114
+ - One task.
115
+ - One agent per match.
116
+ - No scores published yet.
117
+
118
+ ---
119
+
120
+ SuperGaming. Apache-2.0.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
app.py CHANGED
@@ -262,58 +262,50 @@ CSS = """
262
 
263
  HERO = """
264
  <div id="pr-hero">
265
- <h1>A live battle royale, as six MCP tools</h1>
266
- <p class="pr-sub">An agent is dropped into a real match with a shrinking safe circle. It is
267
- given its own coordinates and the circle&rsquo;s &mdash; never a direction. Working out
268
- where to go is the thing being measured.</p>
269
  <div class="pr-chips">
270
- <span class="pr-chip on">frozen between tool calls</span>
271
  <span class="pr-chip">six tools</span>
272
- <span class="pr-chip">graded from a snapshot, not the transcript</span>
273
- <span class="pr-chip">Archipelago-compatible</span>
274
  <span class="pr-chip">Apache-2.0</span>
275
  </div>
276
  </div>
277
  """
278
 
279
  WHY = """
280
- <div class="pr-lab">A reasoning test, not a reaction test</div>
281
 
282
- **The match is frozen between tool calls.** The simulation advances only inside a tool that
283
- says it does, by exactly the amount it asks for. A model may think for thirty seconds and the
284
- game is in precisely the state it left it &mdash; a slow model is not punished and a fast one
285
- is not rewarded. The same seed and the same calls reproduce the same match. Almost no
286
- real-time game can be evaluated this way.
287
 
288
- <div class="pr-lab" style="margin-top:20px">What the agent is not given</div>
289
 
290
- There is no `move_to(x, z)` and no &ldquo;the circle is 40&deg; to your left&rdquo; field.
291
- `get_situation` returns raw coordinates. A server that pre-computed the bearing would score
292
- every model identically and measure nothing. Movement is body-relative for the same reason.
293
  """
294
 
295
  NOTE = """
296
- <div class="pr-lab">How it is graded</div>
297
 
298
- Every call that advances the match rewrites `outcome.json` from the game&rsquo;s own state.
299
- The score never comes from anything the agent said, so an agent cannot talk its way to a
300
- pass &mdash; and a run that crashed or gave up is still graded on what it actually did.
301
- A reference eval definition and the Archipelago task pack are in the Files tab.
302
 
303
  <div class="pr-lab" style="margin-top:20px">What this page runs</div>
304
 
305
- The tools, schemas, scoring and outcome file are the real ones, and the map is drawn only
306
- from what the tools reported &mdash; you see exactly what the agent saw. The **game** is
307
- not: a Space has no Unity server, so the tools drive a stand-in that speaks the same bridge
308
- protocol and reproduces the real game&rsquo;s response characteristics. For the real game,
309
- the environment is hosted.
310
  """
311
 
312
  FOOT = """
313
  <div id="pr-foot">
314
- Built by SuperGaming from the live build of Prime Rush &nbsp;&middot;&nbsp;
315
- adapter, task pack, reference grader and tests are in the Files tab &nbsp;&middot;&nbsp;
316
- Apache-2.0
317
  </div>
318
  """
319
 
 
262
 
263
  HERO = """
264
  <div id="pr-hero">
265
+ <h1>Prime Rush as an MCP environment</h1>
266
+ <p class="pr-sub">An agent plays a battle-royale match through six tools. A safe circle
267
+ shrinks and the agent has to be inside it. It is given its own coordinates and the
268
+ circle&rsquo;s, and has to work out the direction itself.</p>
269
  <div class="pr-chips">
 
270
  <span class="pr-chip">six tools</span>
271
+ <span class="pr-chip">runs under Archipelago</span>
 
272
  <span class="pr-chip">Apache-2.0</span>
273
  </div>
274
  </div>
275
  """
276
 
277
  WHY = """
278
+ <div class="pr-lab">Timing</div>
279
 
280
+ The match only advances inside a tool call, by the amount that call asks for. A model can
281
+ take as long as it likes to decide and the game waits. The same seed and the same calls
282
+ give the same match.
 
 
283
 
284
+ <div class="pr-lab" style="margin-top:20px">What the agent gets</div>
285
 
286
+ `get_situation` returns coordinates: where it is, which way it is facing, where the circle
287
+ is and how big. There is no `move_to` and no bearing. Movement is relative to the direction
288
+ it is facing, so it has to turn first.
289
  """
290
 
291
  NOTE = """
292
+ <div class="pr-lab">Scoring</div>
293
 
294
+ Each call that advances the match writes `outcome.json` from the game state. The score is
295
+ read from that file, not from the transcript, and a run that stops early is still scored on
296
+ what it did. The task pack and a reference grader are in the Files tab.
 
297
 
298
  <div class="pr-lab" style="margin-top:20px">What this page runs</div>
299
 
300
+ The tools, their schemas and the scoring are the ones a lab would run. The game is not: a
301
+ Space has no Unity server, so the tools drive a stand-in that speaks the same protocol. The
302
+ map shows only what the tools returned.
 
 
303
  """
304
 
305
  FOOT = """
306
  <div id="pr-foot">
307
+ SuperGaming &nbsp;&middot;&nbsp; code, task pack and tests in the Files tab
308
+ &nbsp;&middot;&nbsp; Apache-2.0
 
309
  </div>
310
  """
311