E-Rong commited on
Commit
6dc6669
Β·
verified Β·
1 Parent(s): 47c41e4

push phase 3 training

Browse files
This view is limited to 50 files because it contains too many changes. Β  See raw diff
Files changed (50) hide show
  1. data/phase2_final.zip +3 -0
  2. data/phase3_ckpt_1050000.zip +3 -0
  3. data/phase3_ckpt_1100000.zip +3 -0
  4. data/phase3_ckpt_1150000.zip +3 -0
  5. data/phase3_ckpt_1200000.zip +3 -0
  6. data/phase3_ckpt_1250000.zip +3 -0
  7. data/phase3_ckpt_1300000.zip +3 -0
  8. data/phase3_ckpt_1350000.zip +3 -0
  9. data/phase3_ckpt_1400000.zip +3 -0
  10. data/phase3_ckpt_1450000.zip +3 -0
  11. data/phase3_ckpt_1500000.zip +3 -0
  12. data/phase3_ckpt_1550000.zip +3 -0
  13. data/phase3_ckpt_1600000.zip +3 -0
  14. data/phase3_ckpt_1650000.zip +3 -0
  15. phase3_eval_1050000.txt +4 -0
  16. phase3_eval_1100000.txt +4 -0
  17. phase3_eval_1150000.txt +4 -0
  18. phase3_eval_1200000.txt +4 -0
  19. phase3_eval_1250000.txt +4 -0
  20. phase3_eval_1300000.txt +4 -0
  21. phase3_eval_1350000.txt +4 -0
  22. phase3_eval_1400000.txt +4 -0
  23. phase3_eval_1450000.txt +4 -0
  24. phase3_eval_1500000.txt +4 -0
  25. phase3_eval_1550000.txt +4 -0
  26. phase3_eval_1600000.txt +4 -0
  27. til-26-ae-repo/.gitattributes +35 -0
  28. til-26-ae-repo/README.md +14 -0
  29. til-26-ae-repo/ae/Dockerfile +23 -0
  30. til-26-ae-repo/ae/README.md +77 -0
  31. til-26-ae-repo/ae/requirements.txt +10 -0
  32. til-26-ae-repo/ae/src/__init__.py +1 -0
  33. til-26-ae-repo/ae/src/__pycache__/ae_manager.cpython-312.pyc +0 -0
  34. til-26-ae-repo/ae/src/__pycache__/ae_manager.cpython-313.pyc +0 -0
  35. til-26-ae-repo/ae/src/__pycache__/ae_server.cpython-312.pyc +0 -0
  36. til-26-ae-repo/ae/src/__pycache__/ae_server.cpython-313.pyc +0 -0
  37. til-26-ae-repo/ae/src/ae_manager.py +105 -0
  38. til-26-ae-repo/ae/src/ae_server.py +42 -0
  39. til-26-ae-repo/ae/tests/__pycache__/conftest.cpython-312-pytest-9.0.3.pyc +0 -0
  40. til-26-ae-repo/ae/tests/__pycache__/conftest.cpython-313-pytest-9.0.3.pyc +0 -0
  41. til-26-ae-repo/ae/tests/__pycache__/test_ae_api.cpython-312-pytest-9.0.3.pyc +0 -0
  42. til-26-ae-repo/ae/tests/__pycache__/test_ae_api.cpython-313-pytest-9.0.3.pyc +0 -0
  43. til-26-ae-repo/ae/tests/conftest.py +11 -0
  44. til-26-ae-repo/ae/tests/test_ae_api.py +51 -0
  45. til-26-ae-repo/test/test_ae.py +62 -0
  46. til-26-ae-repo/til-26-ae/.gitignore +186 -0
  47. til-26-ae-repo/til-26-ae/.python-version +1 -0
  48. til-26-ae-repo/til-26-ae/AE-with-til_environment.md +270 -0
  49. til-26-ae-repo/til-26-ae/LICENSE +21 -0
  50. til-26-ae-repo/til-26-ae/README.md +175 -0
data/phase2_final.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:afa498308b45deb1ab78260708d761913f71afab6fd479b6f60b2f6892ab1974
3
+ size 3418573
data/phase3_ckpt_1050000.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5001400b53e3c5c7628047424878e12672d343a6e74a3f9e27630149b4d1c2f3
3
+ size 3418561
data/phase3_ckpt_1100000.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b880ccd82ec208405ca2dad4eeaa14d7d3715c6e7adfa731e8d3822fc3d3ce6b
3
+ size 3418560
data/phase3_ckpt_1150000.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4d996e0e4d8d296e1b2da51251e4209d69850ef3118285ac54b1d3130ae4a895
3
+ size 3418560
data/phase3_ckpt_1200000.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d22eec59610d3789c9450043d85da29a065a09b5b26fa87dd4dbb9fc3524fcdb
3
+ size 3418560
data/phase3_ckpt_1250000.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5314c609198a1c6df777d21326da66e0928a967c17fe5fa3d1f888a2485c1213
3
+ size 3418560
data/phase3_ckpt_1300000.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b0155e5f790eddd5594e361377ba1b810661b825b253b474f26123ddbb19e0ad
3
+ size 3418561
data/phase3_ckpt_1350000.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f02cdf8448a0a7b19f7d4b95e90677fd976b013cacba0c5a6fb62ac30f7427f6
3
+ size 3418561
data/phase3_ckpt_1400000.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dcf7d4bba9808c5f6e869114b284022f6ab28627199abcca7e62e042d1d48ecb
3
+ size 3418561
data/phase3_ckpt_1450000.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:895cdda788af09d15563a73a2b0568e1e4af6112eac860fc8809da1d4ecfbed8
3
+ size 3418560
data/phase3_ckpt_1500000.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:edd4e345c28a8cb9f2ab3a81165e42b2c9a4793191f053fdbd892b24f21b8528
3
+ size 3418560
data/phase3_ckpt_1550000.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d3c6effac4444aedc35f8f331b7e346f7be07bc8288f423bb293c78a6d153091
3
+ size 3418561
data/phase3_ckpt_1600000.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d846ab543cd7ca124c587c686e25a8a81359ff63b1b0171dfe780ab397d66065
3
+ size 3418561
data/phase3_ckpt_1650000.zip ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be146b2f5134cd99c1ec555985470fe95a7efb219163cef6eb08601a291b97de
3
+ size 3418561
phase3_eval_1050000.txt ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ Stage: static
2
+ Step: 1050000
3
+ WinRate: 100.0%
4
+ AvgReward: 256.8
phase3_eval_1100000.txt ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ Stage: random
2
+ Step: 1100000
3
+ WinRate: 95.0%
4
+ AvgReward: 205.0
phase3_eval_1150000.txt ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ Stage: simple_bomb
2
+ Step: 1150000
3
+ WinRate: 82.0%
4
+ AvgReward: 106.5
phase3_eval_1200000.txt ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ Stage: evasive
2
+ Step: 1200000
3
+ WinRate: 92.0%
4
+ AvgReward: 167.4
phase3_eval_1250000.txt ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ Stage: mixed
2
+ Step: 1250000
3
+ WinRate: 93.0%
4
+ AvgReward: 170.0
phase3_eval_1300000.txt ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ Stage: mixed
2
+ Step: 1300000
3
+ WinRate: 91.0%
4
+ AvgReward: 157.5
phase3_eval_1350000.txt ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ Stage: mixed
2
+ Step: 1350000
3
+ WinRate: 98.0%
4
+ AvgReward: 216.9
phase3_eval_1400000.txt ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ Stage: mixed
2
+ Step: 1400000
3
+ WinRate: 94.0%
4
+ AvgReward: 174.5
phase3_eval_1450000.txt ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ Stage: mixed
2
+ Step: 1450000
3
+ WinRate: 93.0%
4
+ AvgReward: 205.1
phase3_eval_1500000.txt ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ Stage: mixed
2
+ Step: 1500000
3
+ WinRate: 92.0%
4
+ AvgReward: 205.3
phase3_eval_1550000.txt ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ Stage: mixed
2
+ Step: 1550000
3
+ WinRate: 92.0%
4
+ AvgReward: 172.1
phase3_eval_1600000.txt ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ Stage: mixed
2
+ Step: 1600000
3
+ WinRate: 98.0%
4
+ AvgReward: 205.9
til-26-ae-repo/.gitattributes ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
til-26-ae-repo/README.md ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: Til 26 Ae
3
+ emoji: πŸ”₯
4
+ colorFrom: purple
5
+ colorTo: green
6
+ sdk: gradio
7
+ sdk_version: 6.14.0
8
+ python_version: '3.12'
9
+ app_file: app.py
10
+ pinned: false
11
+ short_description: Autonomous Exploration Challenge
12
+ ---
13
+
14
+ Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
til-26-ae-repo/ae/Dockerfile ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Dockerfile for building the AE image.
2
+
3
+
4
+ # The base image. Using a regular Python base because AE agents don't typically
5
+ # require GPU. This will speed up your evaluation.
6
+ FROM python:3.11-slim
7
+
8
+ # Configures settings for the image.
9
+ ENV PYTHONDONTWRITEBYTECODE=1
10
+ ENV PYTHONUNBUFFERED=1
11
+ ENV PIP_ROOT_USER_ACTION=ignore
12
+ WORKDIR /workspace
13
+
14
+ # Installs your dependencies.
15
+ RUN pip install -U pip
16
+ COPY requirements.txt .
17
+ RUN pip install --no-cache-dir -r requirements.txt
18
+
19
+ # Copies your source files.
20
+ COPY src .
21
+
22
+ # Starts your model server.
23
+ CMD uvicorn ae_server:app --port 5005 --host 0.0.0.0
til-26-ae-repo/ae/README.md ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # AE
2
+
3
+ Your AE challenge is to direct your agent through the game map while interacting with other agents and completing challenges.
4
+
5
+ This Readme provides a brief overview of the interface format; see the Wiki for the full [challenge specifications](https://github.com/til-ai/til-26/wiki/Challenge-specifications).
6
+
7
+ ## Input
8
+
9
+ The input is sent via a POST request to the `/ae` route on port `5005`. It is a JSON object structured as such:
10
+
11
+ ```JSON
12
+ {
13
+ "instances": [
14
+ {
15
+ "observation": {
16
+ "agent_viewcone": [[[0, ...], ...], ...],
17
+ "base_viewcone": [[[0, ...], ...], ...],
18
+ "direction": 0,
19
+ "location": [0, 0],
20
+ "base_location": [0, 0],
21
+ "health": [60.0],
22
+ "frozen_ticks": 0,
23
+ "base_health": [100.0],
24
+ "team_resources": [0.0],
25
+ "team_bombs": 0,
26
+ "step": 0,
27
+ "action_mask": [1, 1, 1, 1, 1, 0]
28
+ }
29
+ }
30
+ ]
31
+ }
32
+ ```
33
+
34
+
35
+ | Field | Shape / type | Description |
36
+ | --- | --- | --- |
37
+ | `agent_viewcone` | `float32 [7 Γ— 5 Γ— 25]` | Viewcone centred on this agent, oriented to its facing direction |
38
+ | `base_viewcone` | `float32 [5 Γ— 5 Γ— 25]` | Square view centred on the team base |
39
+ | `direction` | `Discrete(4)` | Facing direction (0=RIGHT, 1=DOWN, 2=LEFT, 3=UP) |
40
+ | `location` | `uint8 [2]` | Agent (x, y) grid position |
41
+ | `base_location` | `uint8 [2]` | Team base (x, y) grid position |
42
+ | `health` | `float32 [1]` | Agent current HP |
43
+ | `frozen_ticks` | `Discrete(freeze_turns+1)` | Remaining freeze steps (0 = active) |
44
+ | `base_health` | `float32 [1]` | Team base current HP |
45
+ | `team_resources` | `float32 [1]` | Accumulated resource ratio for this agent's team |
46
+ | `team_bombs` | `Discrete(max_team_bombs+1)` | Bomb stockpile for this agent's team |
47
+ | `step` | `Discrete(num_iters+1)` | Current step index |
48
+ | `action_mask` | `uint8 [6]` | Binary mask β€” 1 = action is legal this step |
49
+
50
+ The length of the `instances` array is 1.
51
+
52
+ NOTE: Reset as a POST endpoint on your ae_server.py will not be called. You are recommended to check if the current observation step is 0, and if so, reset your system internally. We will NOT be calling /reset to your server, for qualifiers OR finals.
53
+
54
+ ## Output
55
+
56
+ Your route handler function must return a `dict` with this structure:
57
+
58
+ ```Python
59
+ {
60
+ "predictions": [
61
+ {
62
+ "action": 0
63
+ }
64
+ ]
65
+ }
66
+ ```
67
+
68
+ The action is an integer:
69
+
70
+ | Index | Name | Description |
71
+ | --- | --- | --- |
72
+ | 0 | `FORWARD` | Move one cell in the facing direction |
73
+ | 1 | `BACKWARD` | Move one cell opposite to facing direction |
74
+ | 2 | `LEFT` | Turn 90Β° counter-clockwise |
75
+ | 3 | `RIGHT` | Turn 90Β° clockwise |
76
+ | 4 | `STAY` | Do not move |
77
+ | 5 | `PLACE_BOMB` | Place a bomb at the current cell (requires `team_bombs > 0`) |
til-26-ae-repo/ae/requirements.txt ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ # Dependencies for AE.
2
+ fastapi
3
+ uvicorn[standard]
4
+
5
+ # Model inference dependencies
6
+ sb3-contrib
7
+ stable-baselines3
8
+ numpy
9
+ torch --index-url https://download.pytorch.org/whl/cpu
10
+ huggingface_hub
til-26-ae-repo/ae/src/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ from . import ae_manager, ae_server
til-26-ae-repo/ae/src/__pycache__/ae_manager.cpython-312.pyc ADDED
Binary file (1.03 kB). View file
 
til-26-ae-repo/ae/src/__pycache__/ae_manager.cpython-313.pyc ADDED
Binary file (1.03 kB). View file
 
til-26-ae-repo/ae/src/__pycache__/ae_server.cpython-312.pyc ADDED
Binary file (1.82 kB). View file
 
til-26-ae-repo/ae/src/__pycache__/ae_server.cpython-313.pyc ADDED
Binary file (1.82 kB). View file
 
til-26-ae-repo/ae/src/ae_manager.py ADDED
@@ -0,0 +1,105 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """AE Manager β€” loads trained MaskablePPO and serves inference for Bomberman."""
2
+
3
+ import os
4
+ import numpy as np
5
+ from sb3_contrib import MaskablePPO
6
+ from huggingface_hub import hf_hub_download
7
+
8
+ # Model repo where checkpoints are pushed during training
9
+ MODEL_REPO = "E-Rong/til-26-ae-agent"
10
+
11
+ # Obs flattening constants (must match training script exactly)
12
+ _AGENT_VIEWCONE_SIZE = 7 * 5 * 25 # 875
13
+ _BASE_VIEWCONE_SIZE = 5 * 5 * 25 # 625
14
+ _SCALAR_SIZE = 11 # direction + location[2] + base_location[2] + health + frozen_ticks + base_health + team_resources + team_bombs + step
15
+ TOTAL_OBS_SIZE = _AGENT_VIEWCONE_SIZE + _BASE_VIEWCONE_SIZE + _SCALAR_SIZE # 1511
16
+
17
+
18
+ def _flatten_obs(obs_dict: dict) -> np.ndarray:
19
+ """Flatten a Bomberman observation dict into the (1511,) vector used at train time."""
20
+ return np.concatenate([
21
+ np.array(obs_dict["agent_viewcone"], dtype=np.float32).flatten(),
22
+ np.array(obs_dict["base_viewcone"], dtype=np.float32).flatten(),
23
+ np.array([obs_dict["direction"]], dtype=np.float32),
24
+ np.array(obs_dict["location"], dtype=np.float32).flatten(),
25
+ np.array(obs_dict["base_location"], dtype=np.float32).flatten(),
26
+ np.array(obs_dict["health"], dtype=np.float32).flatten(),
27
+ np.array([obs_dict["frozen_ticks"]], dtype=np.float32),
28
+ np.array(obs_dict["base_health"], dtype=np.float32).flatten(),
29
+ np.array(obs_dict["team_resources"], dtype=np.float32).flatten(),
30
+ np.array([obs_dict["team_bombs"]], dtype=np.float32),
31
+ np.array([obs_dict["step"]], dtype=np.float32),
32
+ ], dtype=np.float32)
33
+
34
+
35
+ class AEManager:
36
+ """Loads a trained MaskablePPO model from the Hub and serves actions."""
37
+
38
+ def __init__(self):
39
+ self.model = None
40
+ self._last_step = -1
41
+
42
+ # Candidates in priority order β€” most trained first
43
+ candidates = [
44
+ "phase3_final.zip",
45
+ "phase2_final.zip",
46
+ "phase1_final.zip",
47
+ "ckpt_200000.zip",
48
+ "ckpt_150000.zip",
49
+ "ckpt_100000.zip",
50
+ "ckpt_50000.zip",
51
+ ]
52
+
53
+ for name in candidates:
54
+ try:
55
+ path = hf_hub_download(
56
+ repo_id=MODEL_REPO,
57
+ filename=name,
58
+ repo_type="model",
59
+ local_dir="/workspace/models",
60
+ local_dir_use_symlinks=False,
61
+ )
62
+ self.model = MaskablePPO.load(path)
63
+ print(f"[AEManager] Loaded model: {name} from {MODEL_REPO}")
64
+ break
65
+ except Exception as e:
66
+ print(f"[AEManager] {name} not available yet ({e}). Trying next...")
67
+
68
+ if self.model is None:
69
+ print("[AEManager] WARNING: No trained model found β€” will return STAY.")
70
+
71
+ def ae(self, observation: dict) -> int:
72
+ """Return an action integer given the observation dict.
73
+
74
+ Args:
75
+ observation: Dict with keys matching the AE input spec.
76
+
77
+ Returns:
78
+ Integer action in [0, 5].
79
+ """
80
+ # Internal reset tracking (recommended by challenge spec)
81
+ step = observation.get("step", 0)
82
+ if step == 0 and step != self._last_step:
83
+ # New episode started β€” PPO is stateless so nothing to reset,
84
+ # but we can log it for debugging.
85
+ print(f"[AEManager] New episode detected at step 0")
86
+ self._last_step = step
87
+
88
+ # Action mask β€” 1 means legal
89
+ mask = np.array(observation.get("action_mask", [1] * 6), dtype=bool)
90
+ valid = np.where(mask)[0]
91
+
92
+ if self.model is None:
93
+ # Fallback if model is missing: random valid action
94
+ return int(np.random.choice(valid)) if len(valid) > 0 else 4
95
+
96
+ # Flatten observation
97
+ obs_vec = _flatten_obs(observation)
98
+
99
+ # Predict with masking (deterministic = True for evaluation)
100
+ action, _ = self.model.predict(
101
+ obs_vec,
102
+ action_masks=mask,
103
+ deterministic=True,
104
+ )
105
+ return int(action)
til-26-ae-repo/ae/src/ae_server.py ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Runs the AE server."""
2
+
3
+ # Unless you want to do something special with the server, you shouldn't need
4
+ # to change anything in this file.
5
+
6
+
7
+ from ae_manager import AEManager
8
+ from fastapi import FastAPI, Request
9
+
10
+ app = FastAPI()
11
+ manager = AEManager()
12
+
13
+
14
+ @app.post("/ae")
15
+ async def ae(request: Request) -> dict[str, list[dict[str, int]]]:
16
+ """Feeds an observation into the AE model.
17
+
18
+ Returns action taken given current observation (int)
19
+ """
20
+
21
+ # get observation, feed into model
22
+ input_json = await request.json()
23
+
24
+ predictions = []
25
+ # each is a dict with one key "observation" and the value as a dictionary observation
26
+ for instance in input_json["instances"]:
27
+ observation = instance["observation"]
28
+ # reset environment on a new round
29
+ # You will have to do your own internal counting and reset your own system between rounds!
30
+ # if observation["step"] == 0:
31
+ # do internal resetting here
32
+ predictions.append({"action": manager.ae(observation)})
33
+ return {"predictions": predictions}
34
+
35
+
36
+ # ------------------------------ RESET REMOVED ------------------------------
37
+
38
+
39
+ @app.get("/health")
40
+ def health() -> dict[str, str]:
41
+ """Health check function for your model."""
42
+ return {"message": "health ok"}
til-26-ae-repo/ae/tests/__pycache__/conftest.cpython-312-pytest-9.0.3.pyc ADDED
Binary file (796 Bytes). View file
 
til-26-ae-repo/ae/tests/__pycache__/conftest.cpython-313-pytest-9.0.3.pyc ADDED
Binary file (800 Bytes). View file
 
til-26-ae-repo/ae/tests/__pycache__/test_ae_api.cpython-312-pytest-9.0.3.pyc ADDED
Binary file (8.78 kB). View file
 
til-26-ae-repo/ae/tests/__pycache__/test_ae_api.cpython-313-pytest-9.0.3.pyc ADDED
Binary file (9.08 kB). View file
 
til-26-ae-repo/ae/tests/conftest.py ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Expose `ae/src` on sys.path so `ae_server` imports match the Docker image layout."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import sys
6
+ from pathlib import Path
7
+
8
+ _ROOT = Path(__file__).resolve().parents[1]
9
+ _SRC = _ROOT / "src"
10
+ if str(_SRC) not in sys.path:
11
+ sys.path.insert(0, str(_SRC))
til-26-ae-repo/ae/tests/test_ae_api.py ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """In-process HTTP contract tests for the AE FastAPI app (pre-Docker)."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import pytest
6
+ from ae_server import app
7
+ from fastapi.testclient import TestClient
8
+
9
+
10
+ @pytest.fixture
11
+ def client() -> TestClient:
12
+ return TestClient(app)
13
+
14
+
15
+ def _observation(*, step: int) -> dict:
16
+ return {
17
+ "viewcone": [[0, 0], [0, 0]],
18
+ "direction": 0,
19
+ "location": [0, 0],
20
+ "scout": 0,
21
+ "step": step,
22
+ }
23
+
24
+
25
+ def test_health_ok(client: TestClient) -> None:
26
+ r = client.get("/health")
27
+ assert r.status_code == 200
28
+ assert r.json() == {"message": "health ok"}
29
+
30
+
31
+ def test_ae_returns_action(client: TestClient) -> None:
32
+ payload = {"instances": [{"observation": _observation(step=1)}]}
33
+ r = client.post("/ae", json=payload)
34
+ assert r.status_code == 200
35
+ body = r.json()
36
+ assert "predictions" in body
37
+ assert len(body["predictions"]) == 1
38
+ assert body["predictions"][0] == {"action": 0}
39
+
40
+
41
+ def test_ae_step_zero_and_reset(client: TestClient) -> None:
42
+ r = client.post("/ae", json={"instances": [{"observation": _observation(step=0)}]})
43
+ assert r.status_code == 200
44
+ assert r.json()["predictions"][0]["action"] == 0
45
+
46
+ r_reset = client.post("/reset")
47
+ assert r_reset.status_code == 200
48
+
49
+ r2 = client.post("/ae", json={"instances": [{"observation": _observation(step=1)}]})
50
+ assert r2.status_code == 200
51
+ assert isinstance(r2.json()["predictions"][0]["action"], int)
til-26-ae-repo/test/test_ae.py ADDED
@@ -0,0 +1,62 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import json
2
+ import os
3
+
4
+ import requests
5
+ from dotenv import load_dotenv
6
+ from til_environment import bomberman_env
7
+ from til_environment.config import default_config
8
+ from tqdm import tqdm, trange
9
+
10
+ load_dotenv()
11
+ TEAM_NAME = os.getenv("TEAM_NAME")
12
+ TEAM_TRACK = os.getenv("TEAM_TRACK")
13
+
14
+ NUM_ROUNDS = 6
15
+ MAX_SCORE = 1000
16
+
17
+
18
+ def main(novice: bool):
19
+ config = default_config()
20
+ config.env.novice = novice
21
+ env = bomberman_env.basic_env(env_wrappers=[], cfg=config)
22
+ # be the agent at index 0
23
+ _agent = env.possible_agents[0]
24
+ rewards = {agent: 0 for agent in env.possible_agents}
25
+
26
+ for i in trange(NUM_ROUNDS):
27
+ print("Round", i + 1)
28
+ env.reset()
29
+ # reset endpoint
30
+ _ = requests.post("http://localhost:5005/ae")
31
+
32
+ for agent in tqdm(env.agent_iter(), total=1200, leave=False):
33
+ observation, reward, termination, truncation, info = env.last()
34
+ observation = {
35
+ k: v if type(v) in (int, float) else v.tolist()
36
+ for k, v in observation.items()
37
+ }
38
+
39
+ for a in env.agents:
40
+ rewards[a] += env.rewards[a]
41
+
42
+ if termination or truncation:
43
+ action = None
44
+ elif agent == _agent:
45
+ response = requests.post(
46
+ "http://localhost:5005/ae",
47
+ data=json.dumps({"instances": [{"observation": observation}]}),
48
+ )
49
+ predictions = response.json()["predictions"]
50
+
51
+ action = int(predictions[0]["action"])
52
+ else:
53
+ # take random action from other agents
54
+ action = env.action_space(agent).sample()
55
+ env.step(action)
56
+ env.close()
57
+ print(f"total rewards: {rewards[_agent]}")
58
+ print(f"score: {rewards[_agent] / NUM_ROUNDS / MAX_SCORE}")
59
+
60
+
61
+ if __name__ == "__main__":
62
+ main(TEAM_TRACK == "novice")
til-26-ae-repo/til-26-ae/.gitignore ADDED
@@ -0,0 +1,186 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ **/.DS_Store
2
+ .vscode
3
+
4
+ # Byte-compiled / optimized / DLL files
5
+ __pycache__/
6
+ *.py[cod]
7
+ *$py.class
8
+
9
+ # C extensions
10
+ *.so
11
+
12
+ # Distribution / packaging
13
+ .Python
14
+ build/
15
+ develop-eggs/
16
+ dist/
17
+ downloads/
18
+ eggs/
19
+ .eggs/
20
+ lib/
21
+ lib64/
22
+ parts/
23
+ sdist/
24
+ var/
25
+ wheels/
26
+ share/python-wheels/
27
+ *.egg-info/
28
+ .installed.cfg
29
+ *.egg
30
+ MANIFEST
31
+
32
+ # PyInstaller
33
+ # Usually these files are written by a python script from a template
34
+ # before PyInstaller builds the exe, so as to inject date/other infos into it.
35
+ *.manifest
36
+ *.spec
37
+
38
+ # Installer logs
39
+ pip-log.txt
40
+ pip-delete-this-directory.txt
41
+
42
+ # Unit test / coverage reports
43
+ htmlcov/
44
+ .tox/
45
+ .nox/
46
+ .coverage
47
+ .coverage.*
48
+ .cache
49
+ nosetests.xml
50
+ coverage.xml
51
+ *.cover
52
+ *.py,cover
53
+ .hypothesis/
54
+ .pytest_cache/
55
+ cover/
56
+
57
+ # Translations
58
+ *.mo
59
+ *.pot
60
+
61
+ # Django stuff:
62
+ *.log
63
+ local_settings.py
64
+ db.sqlite3
65
+ db.sqlite3-journal
66
+
67
+ # Flask stuff:
68
+ instance/
69
+ .webassets-cache
70
+
71
+ # Scrapy stuff:
72
+ .scrapy
73
+
74
+ # Sphinx documentation
75
+ docs/_build/
76
+
77
+ # PyBuilder
78
+ .pybuilder/
79
+ target/
80
+
81
+ # Jupyter Notebook
82
+ .ipynb_checkpoints
83
+
84
+ # IPython
85
+ profile_default/
86
+ ipython_config.py
87
+
88
+ # pyenv
89
+ # For a library or package, you might want to ignore these files since the code is
90
+ # intended to run in multiple environments; otherwise, check them in:
91
+ # .python-version
92
+
93
+ # pipenv
94
+ # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
95
+ # However, in case of collaboration, if having platform-specific dependencies or dependencies
96
+ # having no cross-platform support, pipenv may install dependencies that don't work, or not
97
+ # install all needed dependencies.
98
+ #Pipfile.lock
99
+
100
+ # UV
101
+ # Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
102
+ # This is especially recommended for binary packages to ensure reproducibility, and is more
103
+ # commonly ignored for libraries.
104
+ #uv.lock
105
+
106
+ # poetry
107
+ # Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
108
+ # This is especially recommended for binary packages to ensure reproducibility, and is more
109
+ # commonly ignored for libraries.
110
+ # https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
111
+ #poetry.lock
112
+
113
+ # pdm
114
+ # Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
115
+ #pdm.lock
116
+ # pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
117
+ # in version control.
118
+ # https://pdm.fming.dev/latest/usage/project/#working-with-version-control
119
+ .pdm.toml
120
+ .pdm-python
121
+ .pdm-build/
122
+
123
+ # PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
124
+ __pypackages__/
125
+
126
+ # Celery stuff
127
+ celerybeat-schedule
128
+ celerybeat.pid
129
+
130
+ # SageMath parsed files
131
+ *.sage.py
132
+
133
+ # Environments
134
+ .env
135
+ .venv
136
+ env/
137
+ venv/
138
+ ENV/
139
+ env.bak/
140
+ venv.bak/
141
+
142
+ # Spyder project settings
143
+ .spyderproject
144
+ .spyproject
145
+
146
+ # Rope project settings
147
+ .ropeproject
148
+
149
+ # mkdocs documentation
150
+ /site
151
+
152
+ # mypy
153
+ .mypy_cache/
154
+ .dmypy.json
155
+ dmypy.json
156
+
157
+ # Pyre type checker
158
+ .pyre/
159
+
160
+ # pytype static type analyzer
161
+ .pytype/
162
+
163
+ # Cython debug symbols
164
+ cython_debug/
165
+
166
+ # PyCharm
167
+ # JetBrains specific template is maintained in a separate JetBrains.gitignore that can
168
+ # be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
169
+ # and can be added to the global gitignore or merged into this file. For a more nuclear
170
+ # option (not recommended) you can uncomment the following to ignore the entire idea folder.
171
+ #.idea/
172
+
173
+ # Ruff stuff:
174
+ .ruff_cache/
175
+
176
+ # PyPI configuration file
177
+ .pypirc
178
+
179
+
180
+ # remove other folders
181
+ til_environment/tests
182
+ til_environment/logs
183
+ noise.ipynb
184
+ multiview.py
185
+ TODOs.md
186
+ OPEN_QUESTIONS.md
til-26-ae-repo/til-26-ae/.python-version ADDED
@@ -0,0 +1 @@
 
 
1
+ 3.10
til-26-ae-repo/til-26-ae/AE-with-til_environment.md ADDED
@@ -0,0 +1,270 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Using the `til_environment` package
2
+
3
+ This page introduces the basics of using the `til_environment` package to run your AE models.
4
+
5
+ **Contents**
6
+ 1. [Using the `til_environment` package](#using-the-til_environment-package)
7
+ 1. [Setup](#setup)
8
+ 2. [Usage \& Running the environment](#usage--running-the-environment)
9
+ 1. [Observation Space](#observation-space)
10
+ 2. [Action space](#action-space)
11
+ 3. [Configuration](#configuration)
12
+ 1. [Key config sections](#key-config-sections)
13
+ 2. [Reward shaping](#reward-shaping)
14
+ 4. [Use during training](#use-during-training)
15
+ 1. [Writing an environment wrapper](#writing-an-environment-wrapper)
16
+ 5. [Package structure](#package-structure)
17
+ 1. [Key concepts](#key-concepts)
18
+
19
+ ---
20
+
21
+ ## Setup
22
+
23
+ **Python 3.10 is required.**
24
+
25
+ Clone the `til-26-ae` repo, then create and activate a virtual environment:
26
+
27
+ ```bash
28
+ python3.10 -m venv .venv
29
+ source .venv/bin/activate # Linux / macOS
30
+ # .venv\Scripts\activate # Windows
31
+ pip install -r requirements.txt
32
+ ```
33
+
34
+ Or, if you prefer [`uv`](https://docs.astral.sh/uv/):
35
+
36
+ ```bash
37
+ uv sync # creates .venv and installs all dependencies automatically. goated
38
+ ```
39
+
40
+ ---
41
+
42
+ ## Usage & Running the environment
43
+
44
+ You can import the raw environment as such:
45
+
46
+ ```python
47
+ from til_environment.bomberman_env import Bomberman
48
+ ```
49
+
50
+ This environment follows the [PettingZoo](https://pettingzoo.farama.org/) AEC (Agent-Environment-Cycle) API. Each call to `env.step()` logs an agent's action; a full round executes after the last agent in the cycle submits its action. As such the overall running loop is functionally identical to that of PettingZoo:
51
+
52
+
53
+ ```python
54
+ from til_environment.bomberman_env import Bomberman
55
+
56
+ env = Bomberman()
57
+
58
+ env.reset(seed=42)
59
+
60
+ for agent in env.agent_iter():
61
+ observation, reward, termination, truncation, info = env.last()
62
+
63
+ if termination or truncation:
64
+ action = None
65
+ else:
66
+ # Insert your policy here. or dont. like why dont you throw random at it :)
67
+ action = env.action_space(agent).sample()
68
+
69
+ env.step(action)
70
+
71
+ env.close()
72
+ ```
73
+
74
+ > **Note:** Termination should never fire off. In this game, your agents have been (unfortunately) cursed with immortality. Recall agents only freeze after reaching 0 health, then respawn where they died. Enjoy playing the full 200 iterations!
75
+
76
+ ### Observation Space
77
+
78
+ We've significantly expanded the observation space from last year. This time, we give a series of binary channels (with significant sparsity, i.e many 0s, few 1s, per channel). For a full description of what each channel represents, see [observation.py](til-26-ae/til_environment/observation.py).
79
+
80
+ | Key | Shape / type | Description |
81
+ | ---------------- | ---------------------------- | ---------------------------------------------------------------------------------------------------------- |
82
+ | `agent_viewcone` | `float32 [7 Γ— 5 Γ— 25]` | Viewcone centred on this agent (widths of 4 ahead, 2 behind, 2 to the side, agent at the center inclusive) |
83
+ | `base_viewcone` | `float32 [S Γ— S Γ— 25]` | Square view centred on this agent's base (S = 2Γ—vision_radius+1) |
84
+ | `direction` | `Discrete(4)` | Agent's facing direction (0=RIGHT, 1=DOWN, 2=LEFT, 3=UP) |
85
+ | `location` | `uint8 [2]` | Agent's (x, y) grid position |
86
+ | `base_location` | `uint8 [2]` | Team base (x, y) grid position |
87
+ | `health` | `float32 [1]` | Agent current HP |
88
+ | `frozen_ticks` | `Discrete(freeze_turns+1)` | Remaining freeze steps (0 = active) |
89
+ | `base_health` | `float32 [1]` | Team base current HP |
90
+ | `team_resources` | `float32 [1]` | Accumulated resource ratio for this agent's team |
91
+ | `team_bombs` | `Discrete(max_team_bombs+1)` | Bomb stockpile for this agent's team |
92
+ | `step` | `Discrete(num_iters+1)` | Current step index |
93
+ | `action_mask` | `uint8 [6]` | Binary mask β€” 1 = action is legal for this agent this step |
94
+
95
+ ### Action space
96
+
97
+ Each agent has 6 discrete actions:
98
+
99
+ | Index | Name | Description |
100
+ | ----- | ------------ | -------------------------------------------------------------------- |
101
+ | 0 | `FORWARD` | Move one cell in the direction the agent is facing |
102
+ | 1 | `BACKWARD` | Move one cell opposite to facing direction |
103
+ | 2 | `LEFT` | Turn 90Β° counter-clockwise |
104
+ | 3 | `RIGHT` | Turn 90Β° clockwise |
105
+ | 4 | `STAY` | Do not move |
106
+ | 5 | `PLACE_BOMB` | Place a bomb at the agent's current cell (requires `team_bombs > 0`) |
107
+
108
+ ---
109
+
110
+ ## Configuration
111
+
112
+ The default config may be obtained via:
113
+
114
+ ```python
115
+ from til_environment.config import default_config
116
+ cfg = default_config()
117
+ ```
118
+
119
+ All tunable environment parameters live in a single OmegaConf config tree. During evaluation for qualifiers, we will use this default configuration: However, you are free to modify it for your training purposes. You can simply modify the YAML file, or do it code-wise as follows:
120
+
121
+ ```python
122
+ from til_environment.config import default_config
123
+ from omegaconf import OmegaConf
124
+
125
+ cfg = default_config()
126
+ # override specific values
127
+ cfg = OmegaConf.merge(cfg, {"env": {"grid_size": 10, "num_teams": 2}})
128
+ # but like why. i love writing more config files
129
+ ```
130
+
131
+ The environment is highly configurable, if you want to explore any ideas, create your own custom reward hooks/events or mutate existing ones, heck even change the environment fundementally for your model training / DP development. Who am I to stop you from trying lots of things out! Not me, probably the time you have. Anyways this ae is quite sandboxable, as long as you acknowledge the default config is what the evaluatons / qualifiers / finals will run.
132
+
133
+ ### Key config sections
134
+
135
+ **`env`**
136
+
137
+ | Key | Default | Description |
138
+ | -------------------- | ------- | ----------------------------------------------------- |
139
+ | `grid_size` | 16 | Grid side length |
140
+ | `num_teams` | 6 | Number of competing teams (one agent + one base each) |
141
+ | `num_iters` | 200 | Maximum steps before truncation |
142
+ | `novice` | `false` | If `true`, fixes the map layout (same every episode) |
143
+ | `render_mode` | `null` | `"human"`, `"rgb_array"`, or `null` |
144
+ | `tile_respawn_steps` | 40 | * Max Steps before a collected tile reappears. Note this is randomly generated across the board based on perlin noise! Read the code / til-26-ae README for more info |
145
+
146
+ **`entities.agent`**
147
+
148
+ | Key | Default | Description |
149
+ | ----------------------- | ------- | ---------------------------------------------- |
150
+ | `health` / `max_health` | 60 | Agent HP |
151
+ | `freeze_turns` | 3 | Iterations the agent is frozen after reaching 0 HP |
152
+
153
+ **`entities.bomb`**
154
+
155
+ | Key | Default | Description |
156
+ | -------------- | ------- | ----------------------------- |
157
+ | `attack` | 20.0 | Damage per blast cell |
158
+ | `blast_radius` | 2 | Radius of explosion |
159
+ | `timer` | 4 | Steps until detonation |
160
+
161
+ **`resources`**
162
+
163
+ | Key | Default | Description |
164
+ | -------------------- | ------- | -------------------------------------------------------------------- |
165
+ | `bomb_cost` | 1.5 | Resource ratio required to convert into one bomb |
166
+ | `base_resource_rate` | 0.1 | Resources added to every team's pool per step (fixed, unconditional) |
167
+ | `starting_bombs` | 3 | Bombs each team starts with |
168
+
169
+ ### Reward shaping
170
+
171
+ All reward values are in the config under `rewards`. As you may notice, several are left as 0: We do not penalize these actions, but leave them as open options for reward shaping (that is within your own training pipeline, modifying them in hopes of attaining different behaviour).
172
+
173
+
174
+ | Key | Default | Credited to | Description |
175
+ | --------------------- | -------------------------- | ----------------------------- | --------------------------------------------------------------------------------------------- |
176
+ | `agent_collide_wall` | 0 | Agent | Agent attempts to move into a wall |
177
+ | `agent_collide_agent` | 0 | Agent | Agent attempts to move into another agent |
178
+ | `collect_mission` | 5.0 | Agent | Agent steps onto a mission tile |
179
+ | `collect_recon` | 1.0 | Agent | Agent steps onto a recon tile |
180
+ | `collect_resource` | 2.0 | Agent | Agent steps onto a resource tile |
181
+ | `attack_damage` | 1.0Γ— damage dealt (20 pts) | Attacker (-1.0Γ— for Defender) | Bomb damages an agent or base; defender receives the equal-and-opposite penalty automatically |
182
+ | `attack_kill` | 15.0 | Attacker (flat) | Bomb reduces an agent to 0 HP |
183
+ | `destroy_wall` | 0 | Attacker | Bomb destroys a destructible wall |
184
+ | `destroy_enemy_base` | 50.0 | Attacker | Bomb destroys an enemy base |
185
+ | `own_base_destroyed` | -50.0 | Defending agent | This agent's base is destroyed |
186
+ | `step_penalty` | 0 | Agent | Applied every step |
187
+ | `stationary_penalty` | 0 | Agent | Agent chose `STAY` |
188
+ | `invalid_action` | 0 | Agent | Agent submitted a masked action |
189
+
190
+ > **Defenders are penalised automatically.** You do not need a separate config entry. For every `attack_damage` event, the damaged entity's controlling agent receives `-attack_damage Γ— damage_dealt` in the same step.
191
+
192
+ ---
193
+
194
+ ## Use during training
195
+
196
+ ### Writing an environment wrapper
197
+
198
+ Custom wrappers inherit from `BaseWrapper` in PettingZoo. You may want to write custom wrappers to modify the way observations are returned. A basic form of wrapping can be observed in the [basic_env function](til-26-ae/til_environment/bomberman_env.py)
199
+
200
+ ```python
201
+ import functools
202
+ from pettingzoo.utils.env import ActionType, AECEnv, AgentID, ObsType
203
+ from pettingzoo.utils.wrappers.base import BaseWrapper
204
+
205
+ class CustomWrapper(BaseWrapper[AgentID, ObsType, ActionType]):
206
+ def __init__(self, env: AECEnv[AgentID, ObsType, ActionType]):
207
+ super().__init__(env)
208
+
209
+ def reset(self, seed=None, options=None):
210
+ super().reset(seed, options)
211
+
212
+ def step(self, action: ActionType):
213
+ super().step(action)
214
+
215
+ def observe(self, agent: AgentID) -> ObsType | None:
216
+ obs = super().observe(agent)
217
+ # modify obs here
218
+ return obs
219
+
220
+ @functools.lru_cache(maxsize=None)
221
+ def observation_space(self, agent):
222
+ return super().observation_space(agent)
223
+ ```
224
+ Then simply wrap it around by passing the environment inside it. A basic example with some existing wrappers can be found at
225
+
226
+ For more wrappers see [PettingZoo Wrappers](https://pettingzoo.farama.org/api/wrappers/pz_wrappers/) and [SuperSuit](https://github.com/Farama-Foundation/SuperSuit).
227
+
228
+ ---
229
+
230
+ ## Package structure
231
+
232
+ ```
233
+ til_environment/
234
+ β”œβ”€β”€ bomberman_env.py # Top-level PettingZoo AECEnv; basic_env() factory lives here
235
+ β”œβ”€β”€ dynamics.py # All game logic: phases, rewards, registry access
236
+ β”œβ”€β”€ arena.py # Maze generation (structural + destructible walls, tile placement)
237
+ β”œβ”€β”€ config.py # OmegaConf-based config tree; load_config() / default_config()
238
+ β”œβ”€β”€ observation.py # Build per-agent viewcone tensors; ViewChannel enum (25 channels)
239
+ β”œβ”€β”€ actions.py # ActionV2 enum, ActionMask (legal-move computation)
240
+ β”œβ”€β”€ types.py # Direction, Tile enums and shared type aliases
241
+ β”œβ”€β”€ renderer.py # Pygame renderer; HUD, replay recording
242
+ β”œβ”€β”€ flatten_dict.py # FlattenDictWrapper β€” flattens obs dict to 1-D array
243
+ β”œβ”€β”€ helpers.py # Miscellaneous utilities
244
+ β”œβ”€β”€ play.py # Interactive harness: launch with `uv run python -m til_environment.play`
245
+ β”‚
246
+ β”œβ”€β”€ entities/
247
+ β”‚ β”œβ”€β”€ base.py # Entity base class (entity_id, team, position, alive)
248
+ β”‚ β”œβ”€β”€ dynamic.py # Agent, Base, Bomb classes
249
+ β”‚ β”œβ”€β”€ static.py # Mission, Recon, Resource, DestructibleWall, AttackIntent
250
+ β”‚ β”œβ”€β”€ protocols.py # Structural interfaces: Attacker, Defender, Vision, Item, Timed, ExternalSideEffect
251
+ β”‚ β”œοΏ½οΏ½β”€ geometry.py # Blast/vision geometry helpers (RadiusAttack, SquareVision)
252
+ β”‚ └── registry.py # EntityRegistry β€” indexed container for all live entities
253
+ β”‚
254
+ β”œβ”€β”€ events/
255
+ β”œβ”€β”€ base.py # EmitRule dataclass + wrap_multi() β€” hooks entity methods to reward events
256
+ └── rewards.py # Rewards accumulator (step + episode); award() method
257
+
258
+ ```
259
+
260
+ ### Key concepts
261
+
262
+ **Agents and teams.** There is exactly one agent per team. Agent IDs are `"agent_0"`, `"agent_1"`, … `"agent_{N-1}"`. Each team also has one base.
263
+
264
+ **Death and respawn.** When an agent's HP reaches 0 it enters a *frozen* state for 3 steps β€” it stays on the grid but its entire action mask is zeroed. After the freeze it respawns at the same tile at full HP. Agents are never removed from the game; only step truncation ends an episode.
265
+
266
+ **Resource economy.** Each team has a shared resource pool (`team_resources`) and bomb stockpile (`team_bombs`). Resources accumulate passively every step at a fixed rate (`0.1`), and are also collected by stepping on resource tiles (`0.5`). Whenever `team_resources β‰₯ 1.5` the surplus auto-converts into bombs.
267
+
268
+ **Bombs.** A placed bomb counts down for `timer` steps then detonates in a cross-shaped blast of square radius `2` (so 2 + 1 + 2, where 1 is the center where the bomb is place). The blast damages agents, bases, and destroys destructible walls in range. Indestructible walls block the blast.
269
+
270
+ **Observation channels.** The viewcone tensor has 25 channels encoding visibility, wall edges (structural and destructible, per direction), tile types, ally/enemy entities, bomb presence and timers, and HP levels. Once more, see [`observation.py`](til-26-ae/til_environment/observation.py) (`ViewChannel` enum) for the full list.
til-26-ae-repo/til-26-ae/LICENSE ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2025 BrainHack TIL-AI
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
til-26-ae-repo/til-26-ae/README.md ADDED
@@ -0,0 +1,175 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # TIL Bomberman β€” Environment
2
+
3
+ ## Quick Start
4
+
5
+ `play.py` controls: `W/A/S/D` or arrow keys to move, `B/F` to place a bomb, `SPACE` to stay, `T` to toggle the respawn-timer overlay, `R` to reset, `Q`/`ESC` to quit.
6
+
7
+ ---
8
+
9
+ ## Overview
10
+
11
+ A two-team [PettingZoo](https://pettingzoo.farama.org/) AEC environment built around bomb placement, base defence, and resource management. Each team controls one agent and one stationary base. Agents explore a maze-like arena, collect resource tokens to fund bombs, and destroy the opposing base. Agents are never removed from the game β€” instead, taking lethal damage freezes them in place for a configurable number of ticks before they respawn at full health on the same tile.
12
+
13
+ ---
14
+
15
+ ## Overall Structure
16
+
17
+ ```
18
+ til_environment/
19
+ β”œβ”€β”€ bomberman_env.py β€” PettingZoo AECEnv wrapper (public API)
20
+ β”œβ”€β”€ dynamics.py β€” all mutable game state and phase logic
21
+ β”œβ”€β”€ arena.py β€” maze generation, tile placement, spawn assignment
22
+ β”œβ”€β”€ observation.py β€” multi-channel viewcone builders
23
+ β”œβ”€β”€ actions.py β€” Action enum, ActionMask builder
24
+ β”œβ”€β”€ entities/
25
+ β”‚ β”œβ”€β”€ dynamic.py β€” Agent, Base, Bomb (mutable entities)
26
+ β”‚ β”œβ”€β”€ static.py β€” Mission, Recon, Resource (collectible entities)
27
+ β”‚ β”œβ”€β”€ protocols.py β€” structural trait protocols (Defender, Attacker, …)
28
+ β”‚ β”œβ”€β”€ geometry.py β€” RadiusAttack blast-cell computation
29
+ β”‚ └── registry.py β€” EntityRegistry (typed, indexed entity container)
30
+ β”œβ”€β”€ renderer.py β€” pygame rendering; never mutates game state
31
+ β”œβ”€β”€ config.py β€” OmegaConf dataclass config hierarchy
32
+ β”œβ”€β”€ bomberman_config.yaml β€” default configuration
33
+ β”œβ”€β”€ helpers.py β€” shared grid utilities
34
+ └── types.py β€” Direction, Tile, Wall enums
35
+ ```
36
+
37
+ **`bomberman_env.py`** is the entry point for external code. It subclasses `AECEnv`, delegates all logic to `Dynamics`, and exposes the standard `reset / step / observe / render` contract. `basic_env()` and `parallel_basic_env()` are convenience factory functions that wrap the raw env with `FlattenDictWrapper` and `frame_stack_v2`.
38
+
39
+ **`dynamics.py`** owns the mutable world state: agent positions and health, team resource and bomb counts, bomb timers, tile respawn queues, and the per-round phase pipeline. It composes `ArenaGenerator`, `ArenaState`, `EntityRegistry`, `ActionMask`, and the `Rewards` system.
40
+
41
+ **`arena.py`** generates the maze (Recursive Backtracking via `mazelib`), marks a fraction of walls as destructible, scatters mission/recon/resource tiles with rotational symmetry across teams, and assigns agent/base starting positions. The wall layout is cached between episodes if the seed does not change. A Perlin-noise + centre-gradient respawn map is generated each episode, giving shorter respawn delays near the map centre.
42
+
43
+ **`entities/`** contains the entity hierarchy. All entities share a common `Entity` base (unique ID, position, team). Behaviour is declared via structural protocols (`Defender`, `Attacker`, `Vision`, `Item`, `Timed`) rather than deep inheritance, so `Dynamics` can dispatch on traits without checking concrete types. `EntityRegistry` is an indexed `dict[str, Entity]` with typed accessors (`agents(team)`, `bases(team)`, `bombs()`, etc.).
44
+
45
+ **`observation.py`** builds the multi-channel float32 viewcone tensors that agents receive. Line-of-sight is computed with a vectorised supercover algorithm (numpy batch array indexing; precomputed per viewcone shape via `lru_cache`).
46
+
47
+ **`renderer.py`** is a stateless pygame drawer. In `"human"` mode it draws to a window; in `"rgb_array"` mode it returns a numpy array. It never reads or writes game state β€” it only reads the `ArenaState`, `EntityRegistry`, and the current `observations` dict passed to it.
48
+
49
+ **`config.py`** defines the full config hierarchy as OmegaConf dataclasses. Every tuneable parameter lives here; no scattered constants anywhere else.
50
+
51
+ ---
52
+
53
+ ## Action Space
54
+
55
+ Each agent has a flat `Discrete(6)` action space. Bases do not act.
56
+
57
+ | Value | Name | Effect |
58
+ |-------|-------------|------------------------------------------------------|
59
+ | 0 | FORWARD | Move one tile in the agent's current facing direction |
60
+ | 1 | BACKWARD | Move one tile opposite to facing |
61
+ | 2 | LEFT | Turn left (rotate facing 90Β°), no movement |
62
+ | 3 | RIGHT | Turn right (rotate facing 90Β°), no movement |
63
+ | 4 | STAY | No action |
64
+ | 5 | PLACE_BOMB | Place a bomb on the agent's current tile |
65
+
66
+ An **action mask** (shape `(6,)`, `uint8`) is included in every observation. Illegal actions are replaced with `STAY` and penalised. Movement is masked when a wall blocks the path (structural or destructible). `PLACE_BOMB` is masked when the team has no bombs available or the agent is frozen.
67
+
68
+ ---
69
+
70
+ ## Observation Space
71
+
72
+ Each agent receives a `Dict` observation:
73
+
74
+ | Key | Space | Description |
75
+ |------------------|-------------------------------------------|--------------------------------------------------|
76
+ | `agent_viewcone` | `Box(0,1, (H,W,25), float32)` | Multi-channel viewcone centred on the agent |
77
+ | `base_viewcone` | `Box(0,1, (B,B,25), float32)` | Square view centred on the team base |
78
+ | `direction` | `Discrete(4)` | Agent's current facing direction |
79
+ | `location` | `Box(0,G, (2,), uint8)` | Agent grid position |
80
+ | `base_location` | `Box(0,G, (2,), uint8)` | Team base grid position |
81
+ | `health` | `Box(0, max_health, (1,), float32)` | Agent HP |
82
+ | `frozen_ticks` | `Discrete(freeze_turns+1)` | Ticks remaining until respawn (0 = active) |
83
+ | `base_health` | `Box(0, max_health, (1,), float32)` | Team base HP |
84
+ | `team_resources` | `Box(0, max_resources, (1,), float32)` | Shared resource pool (bombs cost 1.0 unit each) |
85
+ | `team_bombs` | `Discrete(max_bombs+1)` | Bombs currently available to the team |
86
+ | `step` | `Discrete(num_iters+1)` | Current step index |
87
+ | `action_mask` | `Box(0,1, (6,), uint8)` | Legal actions this step |
88
+
89
+ ### Viewcone channels (25 total)
90
+
91
+ The viewcone is oriented relative to the agent's facing direction. Channel 0 is a line-of-sight flag; channels 1–4 encode wall edges; channels 5–12 encode tile types and entity presence; channels 13–16 flag destructible wall edges; channels 17–24 encode bomb state and entity health ratios. Index by `ViewChannel.*` enum rather than raw integers.
92
+
93
+ ---
94
+
95
+ ## Game Loop and Phases
96
+
97
+ The environment follows the PettingZoo AEC protocol: agents act one at a time. Once all agents have submitted an action the round executes in a fixed phase order:
98
+
99
+ 1. **Validate** β€” illegal actions are replaced with `STAY`; `invalid_action` penalty awarded
100
+ 2. **Place bombs** β€” `PLACE_BOMB` actions spawn bombs; `team_bombs[team]` decremented
101
+ 3. **Movement** β€” agents move or turn; collectibles are picked up inline
102
+ 4. **Detonation** β€” expired bomb timers produce `AttackIntent` lists
103
+ 5. **Damage** β€” intents applied to defenders; agents at 0 HP enter frozen state
104
+ 6. **Upkeep** β€” bomb timers tick; resource pools accrue; auto-convert resources β†’ bombs; frozen agents tick down and respawn
105
+
106
+ Termination fires when a team's base is destroyed. Truncation fires at `num_iters` steps. Frozen (dead) agents alone do **not** trigger termination.
107
+
108
+ ---
109
+
110
+ ## Resource and Bomb Economy
111
+
112
+ Each team maintains a floating-point resource pool (`team_resources`) and an integer bomb count (`team_bombs`). Resources accumulate every tick at `base_resource_rate` as long as the team's base is alive. Picking up a resource tile adds a fixed amount to the pool. When the pool reaches `bomb_cost` (default 1.5), it is decremented and `team_bombs` incremented. Bombs start the episode at `starting_bombs` (default 3).
113
+
114
+ ---
115
+
116
+ ## Configuration
117
+
118
+ All parameters live in `bomberman_config.yaml` (loaded into an OmegaConf `BombermanConfig`). Key sections:
119
+
120
+ ```yaml
121
+ env:
122
+ grid_size: 16
123
+ num_teams: 2
124
+ num_iters: 200
125
+ dynamics:
126
+ vision:
127
+ left: 2
128
+ right: 2
129
+ ahead: 4
130
+ behind: 1
131
+ entities:
132
+ agent:
133
+ max_health: 100.0
134
+ freeze_turns: 3
135
+ base:
136
+ max_health: 200.0
137
+ bomb:
138
+ timer: 5
139
+ blast_radius: 2
140
+ resources:
141
+ bomb_cost: 1.5
142
+ base_resource_rate: 0.1
143
+ starting_bombs: 3
144
+ rewards:
145
+ collect_mission: 5.0
146
+ attack_kill: 10.0
147
+ destroy_enemy_base: 50.0
148
+ ```
149
+
150
+ Load a custom config:
151
+
152
+ ```python
153
+ from til_environment.config import load_config
154
+ from til_environment.bomberman_env import Bomberman
155
+
156
+ env = Bomberman(load_config("my_config.yaml"))
157
+ ```
158
+
159
+ Or use the default:
160
+
161
+ ```python
162
+ from til_environment.bomberman_env import basic_env
163
+
164
+ env = basic_env() # includes FlattenDictWrapper + frame_stack_v2
165
+ ```
166
+
167
+ ---
168
+
169
+ ## Tests
170
+
171
+ ```bash
172
+ uv run pytest til_environment/tests/ -v
173
+ ```
174
+
175
+ The test suite covers bomb placement and detonation, destructible wall behaviour, resource/bomb economy, agent freeze and respawn, termination conditions, action masking, and the Perlin respawn map.