push phase 3 training
Browse filesThis view is limited to 50 files because it contains too many changes. Β See raw diff
- data/phase2_final.zip +3 -0
- data/phase3_ckpt_1050000.zip +3 -0
- data/phase3_ckpt_1100000.zip +3 -0
- data/phase3_ckpt_1150000.zip +3 -0
- data/phase3_ckpt_1200000.zip +3 -0
- data/phase3_ckpt_1250000.zip +3 -0
- data/phase3_ckpt_1300000.zip +3 -0
- data/phase3_ckpt_1350000.zip +3 -0
- data/phase3_ckpt_1400000.zip +3 -0
- data/phase3_ckpt_1450000.zip +3 -0
- data/phase3_ckpt_1500000.zip +3 -0
- data/phase3_ckpt_1550000.zip +3 -0
- data/phase3_ckpt_1600000.zip +3 -0
- data/phase3_ckpt_1650000.zip +3 -0
- phase3_eval_1050000.txt +4 -0
- phase3_eval_1100000.txt +4 -0
- phase3_eval_1150000.txt +4 -0
- phase3_eval_1200000.txt +4 -0
- phase3_eval_1250000.txt +4 -0
- phase3_eval_1300000.txt +4 -0
- phase3_eval_1350000.txt +4 -0
- phase3_eval_1400000.txt +4 -0
- phase3_eval_1450000.txt +4 -0
- phase3_eval_1500000.txt +4 -0
- phase3_eval_1550000.txt +4 -0
- phase3_eval_1600000.txt +4 -0
- til-26-ae-repo/.gitattributes +35 -0
- til-26-ae-repo/README.md +14 -0
- til-26-ae-repo/ae/Dockerfile +23 -0
- til-26-ae-repo/ae/README.md +77 -0
- til-26-ae-repo/ae/requirements.txt +10 -0
- til-26-ae-repo/ae/src/__init__.py +1 -0
- til-26-ae-repo/ae/src/__pycache__/ae_manager.cpython-312.pyc +0 -0
- til-26-ae-repo/ae/src/__pycache__/ae_manager.cpython-313.pyc +0 -0
- til-26-ae-repo/ae/src/__pycache__/ae_server.cpython-312.pyc +0 -0
- til-26-ae-repo/ae/src/__pycache__/ae_server.cpython-313.pyc +0 -0
- til-26-ae-repo/ae/src/ae_manager.py +105 -0
- til-26-ae-repo/ae/src/ae_server.py +42 -0
- til-26-ae-repo/ae/tests/__pycache__/conftest.cpython-312-pytest-9.0.3.pyc +0 -0
- til-26-ae-repo/ae/tests/__pycache__/conftest.cpython-313-pytest-9.0.3.pyc +0 -0
- til-26-ae-repo/ae/tests/__pycache__/test_ae_api.cpython-312-pytest-9.0.3.pyc +0 -0
- til-26-ae-repo/ae/tests/__pycache__/test_ae_api.cpython-313-pytest-9.0.3.pyc +0 -0
- til-26-ae-repo/ae/tests/conftest.py +11 -0
- til-26-ae-repo/ae/tests/test_ae_api.py +51 -0
- til-26-ae-repo/test/test_ae.py +62 -0
- til-26-ae-repo/til-26-ae/.gitignore +186 -0
- til-26-ae-repo/til-26-ae/.python-version +1 -0
- til-26-ae-repo/til-26-ae/AE-with-til_environment.md +270 -0
- til-26-ae-repo/til-26-ae/LICENSE +21 -0
- til-26-ae-repo/til-26-ae/README.md +175 -0
data/phase2_final.zip
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:afa498308b45deb1ab78260708d761913f71afab6fd479b6f60b2f6892ab1974
|
| 3 |
+
size 3418573
|
data/phase3_ckpt_1050000.zip
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5001400b53e3c5c7628047424878e12672d343a6e74a3f9e27630149b4d1c2f3
|
| 3 |
+
size 3418561
|
data/phase3_ckpt_1100000.zip
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b880ccd82ec208405ca2dad4eeaa14d7d3715c6e7adfa731e8d3822fc3d3ce6b
|
| 3 |
+
size 3418560
|
data/phase3_ckpt_1150000.zip
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4d996e0e4d8d296e1b2da51251e4209d69850ef3118285ac54b1d3130ae4a895
|
| 3 |
+
size 3418560
|
data/phase3_ckpt_1200000.zip
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d22eec59610d3789c9450043d85da29a065a09b5b26fa87dd4dbb9fc3524fcdb
|
| 3 |
+
size 3418560
|
data/phase3_ckpt_1250000.zip
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5314c609198a1c6df777d21326da66e0928a967c17fe5fa3d1f888a2485c1213
|
| 3 |
+
size 3418560
|
data/phase3_ckpt_1300000.zip
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b0155e5f790eddd5594e361377ba1b810661b825b253b474f26123ddbb19e0ad
|
| 3 |
+
size 3418561
|
data/phase3_ckpt_1350000.zip
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f02cdf8448a0a7b19f7d4b95e90677fd976b013cacba0c5a6fb62ac30f7427f6
|
| 3 |
+
size 3418561
|
data/phase3_ckpt_1400000.zip
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:dcf7d4bba9808c5f6e869114b284022f6ab28627199abcca7e62e042d1d48ecb
|
| 3 |
+
size 3418561
|
data/phase3_ckpt_1450000.zip
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:895cdda788af09d15563a73a2b0568e1e4af6112eac860fc8809da1d4ecfbed8
|
| 3 |
+
size 3418560
|
data/phase3_ckpt_1500000.zip
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:edd4e345c28a8cb9f2ab3a81165e42b2c9a4793191f053fdbd892b24f21b8528
|
| 3 |
+
size 3418560
|
data/phase3_ckpt_1550000.zip
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d3c6effac4444aedc35f8f331b7e346f7be07bc8288f423bb293c78a6d153091
|
| 3 |
+
size 3418561
|
data/phase3_ckpt_1600000.zip
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d846ab543cd7ca124c587c686e25a8a81359ff63b1b0171dfe780ab397d66065
|
| 3 |
+
size 3418561
|
data/phase3_ckpt_1650000.zip
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:be146b2f5134cd99c1ec555985470fe95a7efb219163cef6eb08601a291b97de
|
| 3 |
+
size 3418561
|
phase3_eval_1050000.txt
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Stage: static
|
| 2 |
+
Step: 1050000
|
| 3 |
+
WinRate: 100.0%
|
| 4 |
+
AvgReward: 256.8
|
phase3_eval_1100000.txt
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Stage: random
|
| 2 |
+
Step: 1100000
|
| 3 |
+
WinRate: 95.0%
|
| 4 |
+
AvgReward: 205.0
|
phase3_eval_1150000.txt
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Stage: simple_bomb
|
| 2 |
+
Step: 1150000
|
| 3 |
+
WinRate: 82.0%
|
| 4 |
+
AvgReward: 106.5
|
phase3_eval_1200000.txt
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Stage: evasive
|
| 2 |
+
Step: 1200000
|
| 3 |
+
WinRate: 92.0%
|
| 4 |
+
AvgReward: 167.4
|
phase3_eval_1250000.txt
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Stage: mixed
|
| 2 |
+
Step: 1250000
|
| 3 |
+
WinRate: 93.0%
|
| 4 |
+
AvgReward: 170.0
|
phase3_eval_1300000.txt
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Stage: mixed
|
| 2 |
+
Step: 1300000
|
| 3 |
+
WinRate: 91.0%
|
| 4 |
+
AvgReward: 157.5
|
phase3_eval_1350000.txt
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Stage: mixed
|
| 2 |
+
Step: 1350000
|
| 3 |
+
WinRate: 98.0%
|
| 4 |
+
AvgReward: 216.9
|
phase3_eval_1400000.txt
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Stage: mixed
|
| 2 |
+
Step: 1400000
|
| 3 |
+
WinRate: 94.0%
|
| 4 |
+
AvgReward: 174.5
|
phase3_eval_1450000.txt
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Stage: mixed
|
| 2 |
+
Step: 1450000
|
| 3 |
+
WinRate: 93.0%
|
| 4 |
+
AvgReward: 205.1
|
phase3_eval_1500000.txt
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Stage: mixed
|
| 2 |
+
Step: 1500000
|
| 3 |
+
WinRate: 92.0%
|
| 4 |
+
AvgReward: 205.3
|
phase3_eval_1550000.txt
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Stage: mixed
|
| 2 |
+
Step: 1550000
|
| 3 |
+
WinRate: 92.0%
|
| 4 |
+
AvgReward: 172.1
|
phase3_eval_1600000.txt
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Stage: mixed
|
| 2 |
+
Step: 1600000
|
| 3 |
+
WinRate: 98.0%
|
| 4 |
+
AvgReward: 205.9
|
til-26-ae-repo/.gitattributes
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
+
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
+
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
+
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
+
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
+
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
+
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
+
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
+
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
+
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
+
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
+
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
+
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
+
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
+
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
+
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 20 |
+
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 21 |
+
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 22 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
+
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
+
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 27 |
+
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
+
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 29 |
+
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 30 |
+
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 31 |
+
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 32 |
+
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
+
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
+
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
til-26-ae-repo/README.md
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: Til 26 Ae
|
| 3 |
+
emoji: π₯
|
| 4 |
+
colorFrom: purple
|
| 5 |
+
colorTo: green
|
| 6 |
+
sdk: gradio
|
| 7 |
+
sdk_version: 6.14.0
|
| 8 |
+
python_version: '3.12'
|
| 9 |
+
app_file: app.py
|
| 10 |
+
pinned: false
|
| 11 |
+
short_description: Autonomous Exploration Challenge
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
|
til-26-ae-repo/ae/Dockerfile
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Dockerfile for building the AE image.
|
| 2 |
+
|
| 3 |
+
|
| 4 |
+
# The base image. Using a regular Python base because AE agents don't typically
|
| 5 |
+
# require GPU. This will speed up your evaluation.
|
| 6 |
+
FROM python:3.11-slim
|
| 7 |
+
|
| 8 |
+
# Configures settings for the image.
|
| 9 |
+
ENV PYTHONDONTWRITEBYTECODE=1
|
| 10 |
+
ENV PYTHONUNBUFFERED=1
|
| 11 |
+
ENV PIP_ROOT_USER_ACTION=ignore
|
| 12 |
+
WORKDIR /workspace
|
| 13 |
+
|
| 14 |
+
# Installs your dependencies.
|
| 15 |
+
RUN pip install -U pip
|
| 16 |
+
COPY requirements.txt .
|
| 17 |
+
RUN pip install --no-cache-dir -r requirements.txt
|
| 18 |
+
|
| 19 |
+
# Copies your source files.
|
| 20 |
+
COPY src .
|
| 21 |
+
|
| 22 |
+
# Starts your model server.
|
| 23 |
+
CMD uvicorn ae_server:app --port 5005 --host 0.0.0.0
|
til-26-ae-repo/ae/README.md
ADDED
|
@@ -0,0 +1,77 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# AE
|
| 2 |
+
|
| 3 |
+
Your AE challenge is to direct your agent through the game map while interacting with other agents and completing challenges.
|
| 4 |
+
|
| 5 |
+
This Readme provides a brief overview of the interface format; see the Wiki for the full [challenge specifications](https://github.com/til-ai/til-26/wiki/Challenge-specifications).
|
| 6 |
+
|
| 7 |
+
## Input
|
| 8 |
+
|
| 9 |
+
The input is sent via a POST request to the `/ae` route on port `5005`. It is a JSON object structured as such:
|
| 10 |
+
|
| 11 |
+
```JSON
|
| 12 |
+
{
|
| 13 |
+
"instances": [
|
| 14 |
+
{
|
| 15 |
+
"observation": {
|
| 16 |
+
"agent_viewcone": [[[0, ...], ...], ...],
|
| 17 |
+
"base_viewcone": [[[0, ...], ...], ...],
|
| 18 |
+
"direction": 0,
|
| 19 |
+
"location": [0, 0],
|
| 20 |
+
"base_location": [0, 0],
|
| 21 |
+
"health": [60.0],
|
| 22 |
+
"frozen_ticks": 0,
|
| 23 |
+
"base_health": [100.0],
|
| 24 |
+
"team_resources": [0.0],
|
| 25 |
+
"team_bombs": 0,
|
| 26 |
+
"step": 0,
|
| 27 |
+
"action_mask": [1, 1, 1, 1, 1, 0]
|
| 28 |
+
}
|
| 29 |
+
}
|
| 30 |
+
]
|
| 31 |
+
}
|
| 32 |
+
```
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
| Field | Shape / type | Description |
|
| 36 |
+
| --- | --- | --- |
|
| 37 |
+
| `agent_viewcone` | `float32 [7 Γ 5 Γ 25]` | Viewcone centred on this agent, oriented to its facing direction |
|
| 38 |
+
| `base_viewcone` | `float32 [5 Γ 5 Γ 25]` | Square view centred on the team base |
|
| 39 |
+
| `direction` | `Discrete(4)` | Facing direction (0=RIGHT, 1=DOWN, 2=LEFT, 3=UP) |
|
| 40 |
+
| `location` | `uint8 [2]` | Agent (x, y) grid position |
|
| 41 |
+
| `base_location` | `uint8 [2]` | Team base (x, y) grid position |
|
| 42 |
+
| `health` | `float32 [1]` | Agent current HP |
|
| 43 |
+
| `frozen_ticks` | `Discrete(freeze_turns+1)` | Remaining freeze steps (0 = active) |
|
| 44 |
+
| `base_health` | `float32 [1]` | Team base current HP |
|
| 45 |
+
| `team_resources` | `float32 [1]` | Accumulated resource ratio for this agent's team |
|
| 46 |
+
| `team_bombs` | `Discrete(max_team_bombs+1)` | Bomb stockpile for this agent's team |
|
| 47 |
+
| `step` | `Discrete(num_iters+1)` | Current step index |
|
| 48 |
+
| `action_mask` | `uint8 [6]` | Binary mask β 1 = action is legal this step |
|
| 49 |
+
|
| 50 |
+
The length of the `instances` array is 1.
|
| 51 |
+
|
| 52 |
+
NOTE: Reset as a POST endpoint on your ae_server.py will not be called. You are recommended to check if the current observation step is 0, and if so, reset your system internally. We will NOT be calling /reset to your server, for qualifiers OR finals.
|
| 53 |
+
|
| 54 |
+
## Output
|
| 55 |
+
|
| 56 |
+
Your route handler function must return a `dict` with this structure:
|
| 57 |
+
|
| 58 |
+
```Python
|
| 59 |
+
{
|
| 60 |
+
"predictions": [
|
| 61 |
+
{
|
| 62 |
+
"action": 0
|
| 63 |
+
}
|
| 64 |
+
]
|
| 65 |
+
}
|
| 66 |
+
```
|
| 67 |
+
|
| 68 |
+
The action is an integer:
|
| 69 |
+
|
| 70 |
+
| Index | Name | Description |
|
| 71 |
+
| --- | --- | --- |
|
| 72 |
+
| 0 | `FORWARD` | Move one cell in the facing direction |
|
| 73 |
+
| 1 | `BACKWARD` | Move one cell opposite to facing direction |
|
| 74 |
+
| 2 | `LEFT` | Turn 90Β° counter-clockwise |
|
| 75 |
+
| 3 | `RIGHT` | Turn 90Β° clockwise |
|
| 76 |
+
| 4 | `STAY` | Do not move |
|
| 77 |
+
| 5 | `PLACE_BOMB` | Place a bomb at the current cell (requires `team_bombs > 0`) |
|
til-26-ae-repo/ae/requirements.txt
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Dependencies for AE.
|
| 2 |
+
fastapi
|
| 3 |
+
uvicorn[standard]
|
| 4 |
+
|
| 5 |
+
# Model inference dependencies
|
| 6 |
+
sb3-contrib
|
| 7 |
+
stable-baselines3
|
| 8 |
+
numpy
|
| 9 |
+
torch --index-url https://download.pytorch.org/whl/cpu
|
| 10 |
+
huggingface_hub
|
til-26-ae-repo/ae/src/__init__.py
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
from . import ae_manager, ae_server
|
til-26-ae-repo/ae/src/__pycache__/ae_manager.cpython-312.pyc
ADDED
|
Binary file (1.03 kB). View file
|
|
|
til-26-ae-repo/ae/src/__pycache__/ae_manager.cpython-313.pyc
ADDED
|
Binary file (1.03 kB). View file
|
|
|
til-26-ae-repo/ae/src/__pycache__/ae_server.cpython-312.pyc
ADDED
|
Binary file (1.82 kB). View file
|
|
|
til-26-ae-repo/ae/src/__pycache__/ae_server.cpython-313.pyc
ADDED
|
Binary file (1.82 kB). View file
|
|
|
til-26-ae-repo/ae/src/ae_manager.py
ADDED
|
@@ -0,0 +1,105 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""AE Manager β loads trained MaskablePPO and serves inference for Bomberman."""
|
| 2 |
+
|
| 3 |
+
import os
|
| 4 |
+
import numpy as np
|
| 5 |
+
from sb3_contrib import MaskablePPO
|
| 6 |
+
from huggingface_hub import hf_hub_download
|
| 7 |
+
|
| 8 |
+
# Model repo where checkpoints are pushed during training
|
| 9 |
+
MODEL_REPO = "E-Rong/til-26-ae-agent"
|
| 10 |
+
|
| 11 |
+
# Obs flattening constants (must match training script exactly)
|
| 12 |
+
_AGENT_VIEWCONE_SIZE = 7 * 5 * 25 # 875
|
| 13 |
+
_BASE_VIEWCONE_SIZE = 5 * 5 * 25 # 625
|
| 14 |
+
_SCALAR_SIZE = 11 # direction + location[2] + base_location[2] + health + frozen_ticks + base_health + team_resources + team_bombs + step
|
| 15 |
+
TOTAL_OBS_SIZE = _AGENT_VIEWCONE_SIZE + _BASE_VIEWCONE_SIZE + _SCALAR_SIZE # 1511
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
def _flatten_obs(obs_dict: dict) -> np.ndarray:
|
| 19 |
+
"""Flatten a Bomberman observation dict into the (1511,) vector used at train time."""
|
| 20 |
+
return np.concatenate([
|
| 21 |
+
np.array(obs_dict["agent_viewcone"], dtype=np.float32).flatten(),
|
| 22 |
+
np.array(obs_dict["base_viewcone"], dtype=np.float32).flatten(),
|
| 23 |
+
np.array([obs_dict["direction"]], dtype=np.float32),
|
| 24 |
+
np.array(obs_dict["location"], dtype=np.float32).flatten(),
|
| 25 |
+
np.array(obs_dict["base_location"], dtype=np.float32).flatten(),
|
| 26 |
+
np.array(obs_dict["health"], dtype=np.float32).flatten(),
|
| 27 |
+
np.array([obs_dict["frozen_ticks"]], dtype=np.float32),
|
| 28 |
+
np.array(obs_dict["base_health"], dtype=np.float32).flatten(),
|
| 29 |
+
np.array(obs_dict["team_resources"], dtype=np.float32).flatten(),
|
| 30 |
+
np.array([obs_dict["team_bombs"]], dtype=np.float32),
|
| 31 |
+
np.array([obs_dict["step"]], dtype=np.float32),
|
| 32 |
+
], dtype=np.float32)
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
class AEManager:
|
| 36 |
+
"""Loads a trained MaskablePPO model from the Hub and serves actions."""
|
| 37 |
+
|
| 38 |
+
def __init__(self):
|
| 39 |
+
self.model = None
|
| 40 |
+
self._last_step = -1
|
| 41 |
+
|
| 42 |
+
# Candidates in priority order β most trained first
|
| 43 |
+
candidates = [
|
| 44 |
+
"phase3_final.zip",
|
| 45 |
+
"phase2_final.zip",
|
| 46 |
+
"phase1_final.zip",
|
| 47 |
+
"ckpt_200000.zip",
|
| 48 |
+
"ckpt_150000.zip",
|
| 49 |
+
"ckpt_100000.zip",
|
| 50 |
+
"ckpt_50000.zip",
|
| 51 |
+
]
|
| 52 |
+
|
| 53 |
+
for name in candidates:
|
| 54 |
+
try:
|
| 55 |
+
path = hf_hub_download(
|
| 56 |
+
repo_id=MODEL_REPO,
|
| 57 |
+
filename=name,
|
| 58 |
+
repo_type="model",
|
| 59 |
+
local_dir="/workspace/models",
|
| 60 |
+
local_dir_use_symlinks=False,
|
| 61 |
+
)
|
| 62 |
+
self.model = MaskablePPO.load(path)
|
| 63 |
+
print(f"[AEManager] Loaded model: {name} from {MODEL_REPO}")
|
| 64 |
+
break
|
| 65 |
+
except Exception as e:
|
| 66 |
+
print(f"[AEManager] {name} not available yet ({e}). Trying next...")
|
| 67 |
+
|
| 68 |
+
if self.model is None:
|
| 69 |
+
print("[AEManager] WARNING: No trained model found β will return STAY.")
|
| 70 |
+
|
| 71 |
+
def ae(self, observation: dict) -> int:
|
| 72 |
+
"""Return an action integer given the observation dict.
|
| 73 |
+
|
| 74 |
+
Args:
|
| 75 |
+
observation: Dict with keys matching the AE input spec.
|
| 76 |
+
|
| 77 |
+
Returns:
|
| 78 |
+
Integer action in [0, 5].
|
| 79 |
+
"""
|
| 80 |
+
# Internal reset tracking (recommended by challenge spec)
|
| 81 |
+
step = observation.get("step", 0)
|
| 82 |
+
if step == 0 and step != self._last_step:
|
| 83 |
+
# New episode started β PPO is stateless so nothing to reset,
|
| 84 |
+
# but we can log it for debugging.
|
| 85 |
+
print(f"[AEManager] New episode detected at step 0")
|
| 86 |
+
self._last_step = step
|
| 87 |
+
|
| 88 |
+
# Action mask β 1 means legal
|
| 89 |
+
mask = np.array(observation.get("action_mask", [1] * 6), dtype=bool)
|
| 90 |
+
valid = np.where(mask)[0]
|
| 91 |
+
|
| 92 |
+
if self.model is None:
|
| 93 |
+
# Fallback if model is missing: random valid action
|
| 94 |
+
return int(np.random.choice(valid)) if len(valid) > 0 else 4
|
| 95 |
+
|
| 96 |
+
# Flatten observation
|
| 97 |
+
obs_vec = _flatten_obs(observation)
|
| 98 |
+
|
| 99 |
+
# Predict with masking (deterministic = True for evaluation)
|
| 100 |
+
action, _ = self.model.predict(
|
| 101 |
+
obs_vec,
|
| 102 |
+
action_masks=mask,
|
| 103 |
+
deterministic=True,
|
| 104 |
+
)
|
| 105 |
+
return int(action)
|
til-26-ae-repo/ae/src/ae_server.py
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Runs the AE server."""
|
| 2 |
+
|
| 3 |
+
# Unless you want to do something special with the server, you shouldn't need
|
| 4 |
+
# to change anything in this file.
|
| 5 |
+
|
| 6 |
+
|
| 7 |
+
from ae_manager import AEManager
|
| 8 |
+
from fastapi import FastAPI, Request
|
| 9 |
+
|
| 10 |
+
app = FastAPI()
|
| 11 |
+
manager = AEManager()
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
@app.post("/ae")
|
| 15 |
+
async def ae(request: Request) -> dict[str, list[dict[str, int]]]:
|
| 16 |
+
"""Feeds an observation into the AE model.
|
| 17 |
+
|
| 18 |
+
Returns action taken given current observation (int)
|
| 19 |
+
"""
|
| 20 |
+
|
| 21 |
+
# get observation, feed into model
|
| 22 |
+
input_json = await request.json()
|
| 23 |
+
|
| 24 |
+
predictions = []
|
| 25 |
+
# each is a dict with one key "observation" and the value as a dictionary observation
|
| 26 |
+
for instance in input_json["instances"]:
|
| 27 |
+
observation = instance["observation"]
|
| 28 |
+
# reset environment on a new round
|
| 29 |
+
# You will have to do your own internal counting and reset your own system between rounds!
|
| 30 |
+
# if observation["step"] == 0:
|
| 31 |
+
# do internal resetting here
|
| 32 |
+
predictions.append({"action": manager.ae(observation)})
|
| 33 |
+
return {"predictions": predictions}
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
# ------------------------------ RESET REMOVED ------------------------------
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
@app.get("/health")
|
| 40 |
+
def health() -> dict[str, str]:
|
| 41 |
+
"""Health check function for your model."""
|
| 42 |
+
return {"message": "health ok"}
|
til-26-ae-repo/ae/tests/__pycache__/conftest.cpython-312-pytest-9.0.3.pyc
ADDED
|
Binary file (796 Bytes). View file
|
|
|
til-26-ae-repo/ae/tests/__pycache__/conftest.cpython-313-pytest-9.0.3.pyc
ADDED
|
Binary file (800 Bytes). View file
|
|
|
til-26-ae-repo/ae/tests/__pycache__/test_ae_api.cpython-312-pytest-9.0.3.pyc
ADDED
|
Binary file (8.78 kB). View file
|
|
|
til-26-ae-repo/ae/tests/__pycache__/test_ae_api.cpython-313-pytest-9.0.3.pyc
ADDED
|
Binary file (9.08 kB). View file
|
|
|
til-26-ae-repo/ae/tests/conftest.py
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Expose `ae/src` on sys.path so `ae_server` imports match the Docker image layout."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import sys
|
| 6 |
+
from pathlib import Path
|
| 7 |
+
|
| 8 |
+
_ROOT = Path(__file__).resolve().parents[1]
|
| 9 |
+
_SRC = _ROOT / "src"
|
| 10 |
+
if str(_SRC) not in sys.path:
|
| 11 |
+
sys.path.insert(0, str(_SRC))
|
til-26-ae-repo/ae/tests/test_ae_api.py
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""In-process HTTP contract tests for the AE FastAPI app (pre-Docker)."""
|
| 2 |
+
|
| 3 |
+
from __future__ import annotations
|
| 4 |
+
|
| 5 |
+
import pytest
|
| 6 |
+
from ae_server import app
|
| 7 |
+
from fastapi.testclient import TestClient
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
@pytest.fixture
|
| 11 |
+
def client() -> TestClient:
|
| 12 |
+
return TestClient(app)
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
def _observation(*, step: int) -> dict:
|
| 16 |
+
return {
|
| 17 |
+
"viewcone": [[0, 0], [0, 0]],
|
| 18 |
+
"direction": 0,
|
| 19 |
+
"location": [0, 0],
|
| 20 |
+
"scout": 0,
|
| 21 |
+
"step": step,
|
| 22 |
+
}
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
def test_health_ok(client: TestClient) -> None:
|
| 26 |
+
r = client.get("/health")
|
| 27 |
+
assert r.status_code == 200
|
| 28 |
+
assert r.json() == {"message": "health ok"}
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
def test_ae_returns_action(client: TestClient) -> None:
|
| 32 |
+
payload = {"instances": [{"observation": _observation(step=1)}]}
|
| 33 |
+
r = client.post("/ae", json=payload)
|
| 34 |
+
assert r.status_code == 200
|
| 35 |
+
body = r.json()
|
| 36 |
+
assert "predictions" in body
|
| 37 |
+
assert len(body["predictions"]) == 1
|
| 38 |
+
assert body["predictions"][0] == {"action": 0}
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
def test_ae_step_zero_and_reset(client: TestClient) -> None:
|
| 42 |
+
r = client.post("/ae", json={"instances": [{"observation": _observation(step=0)}]})
|
| 43 |
+
assert r.status_code == 200
|
| 44 |
+
assert r.json()["predictions"][0]["action"] == 0
|
| 45 |
+
|
| 46 |
+
r_reset = client.post("/reset")
|
| 47 |
+
assert r_reset.status_code == 200
|
| 48 |
+
|
| 49 |
+
r2 = client.post("/ae", json={"instances": [{"observation": _observation(step=1)}]})
|
| 50 |
+
assert r2.status_code == 200
|
| 51 |
+
assert isinstance(r2.json()["predictions"][0]["action"], int)
|
til-26-ae-repo/test/test_ae.py
ADDED
|
@@ -0,0 +1,62 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import json
|
| 2 |
+
import os
|
| 3 |
+
|
| 4 |
+
import requests
|
| 5 |
+
from dotenv import load_dotenv
|
| 6 |
+
from til_environment import bomberman_env
|
| 7 |
+
from til_environment.config import default_config
|
| 8 |
+
from tqdm import tqdm, trange
|
| 9 |
+
|
| 10 |
+
load_dotenv()
|
| 11 |
+
TEAM_NAME = os.getenv("TEAM_NAME")
|
| 12 |
+
TEAM_TRACK = os.getenv("TEAM_TRACK")
|
| 13 |
+
|
| 14 |
+
NUM_ROUNDS = 6
|
| 15 |
+
MAX_SCORE = 1000
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
def main(novice: bool):
|
| 19 |
+
config = default_config()
|
| 20 |
+
config.env.novice = novice
|
| 21 |
+
env = bomberman_env.basic_env(env_wrappers=[], cfg=config)
|
| 22 |
+
# be the agent at index 0
|
| 23 |
+
_agent = env.possible_agents[0]
|
| 24 |
+
rewards = {agent: 0 for agent in env.possible_agents}
|
| 25 |
+
|
| 26 |
+
for i in trange(NUM_ROUNDS):
|
| 27 |
+
print("Round", i + 1)
|
| 28 |
+
env.reset()
|
| 29 |
+
# reset endpoint
|
| 30 |
+
_ = requests.post("http://localhost:5005/ae")
|
| 31 |
+
|
| 32 |
+
for agent in tqdm(env.agent_iter(), total=1200, leave=False):
|
| 33 |
+
observation, reward, termination, truncation, info = env.last()
|
| 34 |
+
observation = {
|
| 35 |
+
k: v if type(v) in (int, float) else v.tolist()
|
| 36 |
+
for k, v in observation.items()
|
| 37 |
+
}
|
| 38 |
+
|
| 39 |
+
for a in env.agents:
|
| 40 |
+
rewards[a] += env.rewards[a]
|
| 41 |
+
|
| 42 |
+
if termination or truncation:
|
| 43 |
+
action = None
|
| 44 |
+
elif agent == _agent:
|
| 45 |
+
response = requests.post(
|
| 46 |
+
"http://localhost:5005/ae",
|
| 47 |
+
data=json.dumps({"instances": [{"observation": observation}]}),
|
| 48 |
+
)
|
| 49 |
+
predictions = response.json()["predictions"]
|
| 50 |
+
|
| 51 |
+
action = int(predictions[0]["action"])
|
| 52 |
+
else:
|
| 53 |
+
# take random action from other agents
|
| 54 |
+
action = env.action_space(agent).sample()
|
| 55 |
+
env.step(action)
|
| 56 |
+
env.close()
|
| 57 |
+
print(f"total rewards: {rewards[_agent]}")
|
| 58 |
+
print(f"score: {rewards[_agent] / NUM_ROUNDS / MAX_SCORE}")
|
| 59 |
+
|
| 60 |
+
|
| 61 |
+
if __name__ == "__main__":
|
| 62 |
+
main(TEAM_TRACK == "novice")
|
til-26-ae-repo/til-26-ae/.gitignore
ADDED
|
@@ -0,0 +1,186 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
**/.DS_Store
|
| 2 |
+
.vscode
|
| 3 |
+
|
| 4 |
+
# Byte-compiled / optimized / DLL files
|
| 5 |
+
__pycache__/
|
| 6 |
+
*.py[cod]
|
| 7 |
+
*$py.class
|
| 8 |
+
|
| 9 |
+
# C extensions
|
| 10 |
+
*.so
|
| 11 |
+
|
| 12 |
+
# Distribution / packaging
|
| 13 |
+
.Python
|
| 14 |
+
build/
|
| 15 |
+
develop-eggs/
|
| 16 |
+
dist/
|
| 17 |
+
downloads/
|
| 18 |
+
eggs/
|
| 19 |
+
.eggs/
|
| 20 |
+
lib/
|
| 21 |
+
lib64/
|
| 22 |
+
parts/
|
| 23 |
+
sdist/
|
| 24 |
+
var/
|
| 25 |
+
wheels/
|
| 26 |
+
share/python-wheels/
|
| 27 |
+
*.egg-info/
|
| 28 |
+
.installed.cfg
|
| 29 |
+
*.egg
|
| 30 |
+
MANIFEST
|
| 31 |
+
|
| 32 |
+
# PyInstaller
|
| 33 |
+
# Usually these files are written by a python script from a template
|
| 34 |
+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
|
| 35 |
+
*.manifest
|
| 36 |
+
*.spec
|
| 37 |
+
|
| 38 |
+
# Installer logs
|
| 39 |
+
pip-log.txt
|
| 40 |
+
pip-delete-this-directory.txt
|
| 41 |
+
|
| 42 |
+
# Unit test / coverage reports
|
| 43 |
+
htmlcov/
|
| 44 |
+
.tox/
|
| 45 |
+
.nox/
|
| 46 |
+
.coverage
|
| 47 |
+
.coverage.*
|
| 48 |
+
.cache
|
| 49 |
+
nosetests.xml
|
| 50 |
+
coverage.xml
|
| 51 |
+
*.cover
|
| 52 |
+
*.py,cover
|
| 53 |
+
.hypothesis/
|
| 54 |
+
.pytest_cache/
|
| 55 |
+
cover/
|
| 56 |
+
|
| 57 |
+
# Translations
|
| 58 |
+
*.mo
|
| 59 |
+
*.pot
|
| 60 |
+
|
| 61 |
+
# Django stuff:
|
| 62 |
+
*.log
|
| 63 |
+
local_settings.py
|
| 64 |
+
db.sqlite3
|
| 65 |
+
db.sqlite3-journal
|
| 66 |
+
|
| 67 |
+
# Flask stuff:
|
| 68 |
+
instance/
|
| 69 |
+
.webassets-cache
|
| 70 |
+
|
| 71 |
+
# Scrapy stuff:
|
| 72 |
+
.scrapy
|
| 73 |
+
|
| 74 |
+
# Sphinx documentation
|
| 75 |
+
docs/_build/
|
| 76 |
+
|
| 77 |
+
# PyBuilder
|
| 78 |
+
.pybuilder/
|
| 79 |
+
target/
|
| 80 |
+
|
| 81 |
+
# Jupyter Notebook
|
| 82 |
+
.ipynb_checkpoints
|
| 83 |
+
|
| 84 |
+
# IPython
|
| 85 |
+
profile_default/
|
| 86 |
+
ipython_config.py
|
| 87 |
+
|
| 88 |
+
# pyenv
|
| 89 |
+
# For a library or package, you might want to ignore these files since the code is
|
| 90 |
+
# intended to run in multiple environments; otherwise, check them in:
|
| 91 |
+
# .python-version
|
| 92 |
+
|
| 93 |
+
# pipenv
|
| 94 |
+
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
|
| 95 |
+
# However, in case of collaboration, if having platform-specific dependencies or dependencies
|
| 96 |
+
# having no cross-platform support, pipenv may install dependencies that don't work, or not
|
| 97 |
+
# install all needed dependencies.
|
| 98 |
+
#Pipfile.lock
|
| 99 |
+
|
| 100 |
+
# UV
|
| 101 |
+
# Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
|
| 102 |
+
# This is especially recommended for binary packages to ensure reproducibility, and is more
|
| 103 |
+
# commonly ignored for libraries.
|
| 104 |
+
#uv.lock
|
| 105 |
+
|
| 106 |
+
# poetry
|
| 107 |
+
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
|
| 108 |
+
# This is especially recommended for binary packages to ensure reproducibility, and is more
|
| 109 |
+
# commonly ignored for libraries.
|
| 110 |
+
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
|
| 111 |
+
#poetry.lock
|
| 112 |
+
|
| 113 |
+
# pdm
|
| 114 |
+
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
|
| 115 |
+
#pdm.lock
|
| 116 |
+
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
|
| 117 |
+
# in version control.
|
| 118 |
+
# https://pdm.fming.dev/latest/usage/project/#working-with-version-control
|
| 119 |
+
.pdm.toml
|
| 120 |
+
.pdm-python
|
| 121 |
+
.pdm-build/
|
| 122 |
+
|
| 123 |
+
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
|
| 124 |
+
__pypackages__/
|
| 125 |
+
|
| 126 |
+
# Celery stuff
|
| 127 |
+
celerybeat-schedule
|
| 128 |
+
celerybeat.pid
|
| 129 |
+
|
| 130 |
+
# SageMath parsed files
|
| 131 |
+
*.sage.py
|
| 132 |
+
|
| 133 |
+
# Environments
|
| 134 |
+
.env
|
| 135 |
+
.venv
|
| 136 |
+
env/
|
| 137 |
+
venv/
|
| 138 |
+
ENV/
|
| 139 |
+
env.bak/
|
| 140 |
+
venv.bak/
|
| 141 |
+
|
| 142 |
+
# Spyder project settings
|
| 143 |
+
.spyderproject
|
| 144 |
+
.spyproject
|
| 145 |
+
|
| 146 |
+
# Rope project settings
|
| 147 |
+
.ropeproject
|
| 148 |
+
|
| 149 |
+
# mkdocs documentation
|
| 150 |
+
/site
|
| 151 |
+
|
| 152 |
+
# mypy
|
| 153 |
+
.mypy_cache/
|
| 154 |
+
.dmypy.json
|
| 155 |
+
dmypy.json
|
| 156 |
+
|
| 157 |
+
# Pyre type checker
|
| 158 |
+
.pyre/
|
| 159 |
+
|
| 160 |
+
# pytype static type analyzer
|
| 161 |
+
.pytype/
|
| 162 |
+
|
| 163 |
+
# Cython debug symbols
|
| 164 |
+
cython_debug/
|
| 165 |
+
|
| 166 |
+
# PyCharm
|
| 167 |
+
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
|
| 168 |
+
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
|
| 169 |
+
# and can be added to the global gitignore or merged into this file. For a more nuclear
|
| 170 |
+
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
|
| 171 |
+
#.idea/
|
| 172 |
+
|
| 173 |
+
# Ruff stuff:
|
| 174 |
+
.ruff_cache/
|
| 175 |
+
|
| 176 |
+
# PyPI configuration file
|
| 177 |
+
.pypirc
|
| 178 |
+
|
| 179 |
+
|
| 180 |
+
# remove other folders
|
| 181 |
+
til_environment/tests
|
| 182 |
+
til_environment/logs
|
| 183 |
+
noise.ipynb
|
| 184 |
+
multiview.py
|
| 185 |
+
TODOs.md
|
| 186 |
+
OPEN_QUESTIONS.md
|
til-26-ae-repo/til-26-ae/.python-version
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
3.10
|
til-26-ae-repo/til-26-ae/AE-with-til_environment.md
ADDED
|
@@ -0,0 +1,270 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Using the `til_environment` package
|
| 2 |
+
|
| 3 |
+
This page introduces the basics of using the `til_environment` package to run your AE models.
|
| 4 |
+
|
| 5 |
+
**Contents**
|
| 6 |
+
1. [Using the `til_environment` package](#using-the-til_environment-package)
|
| 7 |
+
1. [Setup](#setup)
|
| 8 |
+
2. [Usage \& Running the environment](#usage--running-the-environment)
|
| 9 |
+
1. [Observation Space](#observation-space)
|
| 10 |
+
2. [Action space](#action-space)
|
| 11 |
+
3. [Configuration](#configuration)
|
| 12 |
+
1. [Key config sections](#key-config-sections)
|
| 13 |
+
2. [Reward shaping](#reward-shaping)
|
| 14 |
+
4. [Use during training](#use-during-training)
|
| 15 |
+
1. [Writing an environment wrapper](#writing-an-environment-wrapper)
|
| 16 |
+
5. [Package structure](#package-structure)
|
| 17 |
+
1. [Key concepts](#key-concepts)
|
| 18 |
+
|
| 19 |
+
---
|
| 20 |
+
|
| 21 |
+
## Setup
|
| 22 |
+
|
| 23 |
+
**Python 3.10 is required.**
|
| 24 |
+
|
| 25 |
+
Clone the `til-26-ae` repo, then create and activate a virtual environment:
|
| 26 |
+
|
| 27 |
+
```bash
|
| 28 |
+
python3.10 -m venv .venv
|
| 29 |
+
source .venv/bin/activate # Linux / macOS
|
| 30 |
+
# .venv\Scripts\activate # Windows
|
| 31 |
+
pip install -r requirements.txt
|
| 32 |
+
```
|
| 33 |
+
|
| 34 |
+
Or, if you prefer [`uv`](https://docs.astral.sh/uv/):
|
| 35 |
+
|
| 36 |
+
```bash
|
| 37 |
+
uv sync # creates .venv and installs all dependencies automatically. goated
|
| 38 |
+
```
|
| 39 |
+
|
| 40 |
+
---
|
| 41 |
+
|
| 42 |
+
## Usage & Running the environment
|
| 43 |
+
|
| 44 |
+
You can import the raw environment as such:
|
| 45 |
+
|
| 46 |
+
```python
|
| 47 |
+
from til_environment.bomberman_env import Bomberman
|
| 48 |
+
```
|
| 49 |
+
|
| 50 |
+
This environment follows the [PettingZoo](https://pettingzoo.farama.org/) AEC (Agent-Environment-Cycle) API. Each call to `env.step()` logs an agent's action; a full round executes after the last agent in the cycle submits its action. As such the overall running loop is functionally identical to that of PettingZoo:
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
```python
|
| 54 |
+
from til_environment.bomberman_env import Bomberman
|
| 55 |
+
|
| 56 |
+
env = Bomberman()
|
| 57 |
+
|
| 58 |
+
env.reset(seed=42)
|
| 59 |
+
|
| 60 |
+
for agent in env.agent_iter():
|
| 61 |
+
observation, reward, termination, truncation, info = env.last()
|
| 62 |
+
|
| 63 |
+
if termination or truncation:
|
| 64 |
+
action = None
|
| 65 |
+
else:
|
| 66 |
+
# Insert your policy here. or dont. like why dont you throw random at it :)
|
| 67 |
+
action = env.action_space(agent).sample()
|
| 68 |
+
|
| 69 |
+
env.step(action)
|
| 70 |
+
|
| 71 |
+
env.close()
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
+
> **Note:** Termination should never fire off. In this game, your agents have been (unfortunately) cursed with immortality. Recall agents only freeze after reaching 0 health, then respawn where they died. Enjoy playing the full 200 iterations!
|
| 75 |
+
|
| 76 |
+
### Observation Space
|
| 77 |
+
|
| 78 |
+
We've significantly expanded the observation space from last year. This time, we give a series of binary channels (with significant sparsity, i.e many 0s, few 1s, per channel). For a full description of what each channel represents, see [observation.py](til-26-ae/til_environment/observation.py).
|
| 79 |
+
|
| 80 |
+
| Key | Shape / type | Description |
|
| 81 |
+
| ---------------- | ---------------------------- | ---------------------------------------------------------------------------------------------------------- |
|
| 82 |
+
| `agent_viewcone` | `float32 [7 Γ 5 Γ 25]` | Viewcone centred on this agent (widths of 4 ahead, 2 behind, 2 to the side, agent at the center inclusive) |
|
| 83 |
+
| `base_viewcone` | `float32 [S Γ S Γ 25]` | Square view centred on this agent's base (S = 2Γvision_radius+1) |
|
| 84 |
+
| `direction` | `Discrete(4)` | Agent's facing direction (0=RIGHT, 1=DOWN, 2=LEFT, 3=UP) |
|
| 85 |
+
| `location` | `uint8 [2]` | Agent's (x, y) grid position |
|
| 86 |
+
| `base_location` | `uint8 [2]` | Team base (x, y) grid position |
|
| 87 |
+
| `health` | `float32 [1]` | Agent current HP |
|
| 88 |
+
| `frozen_ticks` | `Discrete(freeze_turns+1)` | Remaining freeze steps (0 = active) |
|
| 89 |
+
| `base_health` | `float32 [1]` | Team base current HP |
|
| 90 |
+
| `team_resources` | `float32 [1]` | Accumulated resource ratio for this agent's team |
|
| 91 |
+
| `team_bombs` | `Discrete(max_team_bombs+1)` | Bomb stockpile for this agent's team |
|
| 92 |
+
| `step` | `Discrete(num_iters+1)` | Current step index |
|
| 93 |
+
| `action_mask` | `uint8 [6]` | Binary mask β 1 = action is legal for this agent this step |
|
| 94 |
+
|
| 95 |
+
### Action space
|
| 96 |
+
|
| 97 |
+
Each agent has 6 discrete actions:
|
| 98 |
+
|
| 99 |
+
| Index | Name | Description |
|
| 100 |
+
| ----- | ------------ | -------------------------------------------------------------------- |
|
| 101 |
+
| 0 | `FORWARD` | Move one cell in the direction the agent is facing |
|
| 102 |
+
| 1 | `BACKWARD` | Move one cell opposite to facing direction |
|
| 103 |
+
| 2 | `LEFT` | Turn 90Β° counter-clockwise |
|
| 104 |
+
| 3 | `RIGHT` | Turn 90Β° clockwise |
|
| 105 |
+
| 4 | `STAY` | Do not move |
|
| 106 |
+
| 5 | `PLACE_BOMB` | Place a bomb at the agent's current cell (requires `team_bombs > 0`) |
|
| 107 |
+
|
| 108 |
+
---
|
| 109 |
+
|
| 110 |
+
## Configuration
|
| 111 |
+
|
| 112 |
+
The default config may be obtained via:
|
| 113 |
+
|
| 114 |
+
```python
|
| 115 |
+
from til_environment.config import default_config
|
| 116 |
+
cfg = default_config()
|
| 117 |
+
```
|
| 118 |
+
|
| 119 |
+
All tunable environment parameters live in a single OmegaConf config tree. During evaluation for qualifiers, we will use this default configuration: However, you are free to modify it for your training purposes. You can simply modify the YAML file, or do it code-wise as follows:
|
| 120 |
+
|
| 121 |
+
```python
|
| 122 |
+
from til_environment.config import default_config
|
| 123 |
+
from omegaconf import OmegaConf
|
| 124 |
+
|
| 125 |
+
cfg = default_config()
|
| 126 |
+
# override specific values
|
| 127 |
+
cfg = OmegaConf.merge(cfg, {"env": {"grid_size": 10, "num_teams": 2}})
|
| 128 |
+
# but like why. i love writing more config files
|
| 129 |
+
```
|
| 130 |
+
|
| 131 |
+
The environment is highly configurable, if you want to explore any ideas, create your own custom reward hooks/events or mutate existing ones, heck even change the environment fundementally for your model training / DP development. Who am I to stop you from trying lots of things out! Not me, probably the time you have. Anyways this ae is quite sandboxable, as long as you acknowledge the default config is what the evaluatons / qualifiers / finals will run.
|
| 132 |
+
|
| 133 |
+
### Key config sections
|
| 134 |
+
|
| 135 |
+
**`env`**
|
| 136 |
+
|
| 137 |
+
| Key | Default | Description |
|
| 138 |
+
| -------------------- | ------- | ----------------------------------------------------- |
|
| 139 |
+
| `grid_size` | 16 | Grid side length |
|
| 140 |
+
| `num_teams` | 6 | Number of competing teams (one agent + one base each) |
|
| 141 |
+
| `num_iters` | 200 | Maximum steps before truncation |
|
| 142 |
+
| `novice` | `false` | If `true`, fixes the map layout (same every episode) |
|
| 143 |
+
| `render_mode` | `null` | `"human"`, `"rgb_array"`, or `null` |
|
| 144 |
+
| `tile_respawn_steps` | 40 | * Max Steps before a collected tile reappears. Note this is randomly generated across the board based on perlin noise! Read the code / til-26-ae README for more info |
|
| 145 |
+
|
| 146 |
+
**`entities.agent`**
|
| 147 |
+
|
| 148 |
+
| Key | Default | Description |
|
| 149 |
+
| ----------------------- | ------- | ---------------------------------------------- |
|
| 150 |
+
| `health` / `max_health` | 60 | Agent HP |
|
| 151 |
+
| `freeze_turns` | 3 | Iterations the agent is frozen after reaching 0 HP |
|
| 152 |
+
|
| 153 |
+
**`entities.bomb`**
|
| 154 |
+
|
| 155 |
+
| Key | Default | Description |
|
| 156 |
+
| -------------- | ------- | ----------------------------- |
|
| 157 |
+
| `attack` | 20.0 | Damage per blast cell |
|
| 158 |
+
| `blast_radius` | 2 | Radius of explosion |
|
| 159 |
+
| `timer` | 4 | Steps until detonation |
|
| 160 |
+
|
| 161 |
+
**`resources`**
|
| 162 |
+
|
| 163 |
+
| Key | Default | Description |
|
| 164 |
+
| -------------------- | ------- | -------------------------------------------------------------------- |
|
| 165 |
+
| `bomb_cost` | 1.5 | Resource ratio required to convert into one bomb |
|
| 166 |
+
| `base_resource_rate` | 0.1 | Resources added to every team's pool per step (fixed, unconditional) |
|
| 167 |
+
| `starting_bombs` | 3 | Bombs each team starts with |
|
| 168 |
+
|
| 169 |
+
### Reward shaping
|
| 170 |
+
|
| 171 |
+
All reward values are in the config under `rewards`. As you may notice, several are left as 0: We do not penalize these actions, but leave them as open options for reward shaping (that is within your own training pipeline, modifying them in hopes of attaining different behaviour).
|
| 172 |
+
|
| 173 |
+
|
| 174 |
+
| Key | Default | Credited to | Description |
|
| 175 |
+
| --------------------- | -------------------------- | ----------------------------- | --------------------------------------------------------------------------------------------- |
|
| 176 |
+
| `agent_collide_wall` | 0 | Agent | Agent attempts to move into a wall |
|
| 177 |
+
| `agent_collide_agent` | 0 | Agent | Agent attempts to move into another agent |
|
| 178 |
+
| `collect_mission` | 5.0 | Agent | Agent steps onto a mission tile |
|
| 179 |
+
| `collect_recon` | 1.0 | Agent | Agent steps onto a recon tile |
|
| 180 |
+
| `collect_resource` | 2.0 | Agent | Agent steps onto a resource tile |
|
| 181 |
+
| `attack_damage` | 1.0Γ damage dealt (20 pts) | Attacker (-1.0Γ for Defender) | Bomb damages an agent or base; defender receives the equal-and-opposite penalty automatically |
|
| 182 |
+
| `attack_kill` | 15.0 | Attacker (flat) | Bomb reduces an agent to 0 HP |
|
| 183 |
+
| `destroy_wall` | 0 | Attacker | Bomb destroys a destructible wall |
|
| 184 |
+
| `destroy_enemy_base` | 50.0 | Attacker | Bomb destroys an enemy base |
|
| 185 |
+
| `own_base_destroyed` | -50.0 | Defending agent | This agent's base is destroyed |
|
| 186 |
+
| `step_penalty` | 0 | Agent | Applied every step |
|
| 187 |
+
| `stationary_penalty` | 0 | Agent | Agent chose `STAY` |
|
| 188 |
+
| `invalid_action` | 0 | Agent | Agent submitted a masked action |
|
| 189 |
+
|
| 190 |
+
> **Defenders are penalised automatically.** You do not need a separate config entry. For every `attack_damage` event, the damaged entity's controlling agent receives `-attack_damage Γ damage_dealt` in the same step.
|
| 191 |
+
|
| 192 |
+
---
|
| 193 |
+
|
| 194 |
+
## Use during training
|
| 195 |
+
|
| 196 |
+
### Writing an environment wrapper
|
| 197 |
+
|
| 198 |
+
Custom wrappers inherit from `BaseWrapper` in PettingZoo. You may want to write custom wrappers to modify the way observations are returned. A basic form of wrapping can be observed in the [basic_env function](til-26-ae/til_environment/bomberman_env.py)
|
| 199 |
+
|
| 200 |
+
```python
|
| 201 |
+
import functools
|
| 202 |
+
from pettingzoo.utils.env import ActionType, AECEnv, AgentID, ObsType
|
| 203 |
+
from pettingzoo.utils.wrappers.base import BaseWrapper
|
| 204 |
+
|
| 205 |
+
class CustomWrapper(BaseWrapper[AgentID, ObsType, ActionType]):
|
| 206 |
+
def __init__(self, env: AECEnv[AgentID, ObsType, ActionType]):
|
| 207 |
+
super().__init__(env)
|
| 208 |
+
|
| 209 |
+
def reset(self, seed=None, options=None):
|
| 210 |
+
super().reset(seed, options)
|
| 211 |
+
|
| 212 |
+
def step(self, action: ActionType):
|
| 213 |
+
super().step(action)
|
| 214 |
+
|
| 215 |
+
def observe(self, agent: AgentID) -> ObsType | None:
|
| 216 |
+
obs = super().observe(agent)
|
| 217 |
+
# modify obs here
|
| 218 |
+
return obs
|
| 219 |
+
|
| 220 |
+
@functools.lru_cache(maxsize=None)
|
| 221 |
+
def observation_space(self, agent):
|
| 222 |
+
return super().observation_space(agent)
|
| 223 |
+
```
|
| 224 |
+
Then simply wrap it around by passing the environment inside it. A basic example with some existing wrappers can be found at
|
| 225 |
+
|
| 226 |
+
For more wrappers see [PettingZoo Wrappers](https://pettingzoo.farama.org/api/wrappers/pz_wrappers/) and [SuperSuit](https://github.com/Farama-Foundation/SuperSuit).
|
| 227 |
+
|
| 228 |
+
---
|
| 229 |
+
|
| 230 |
+
## Package structure
|
| 231 |
+
|
| 232 |
+
```
|
| 233 |
+
til_environment/
|
| 234 |
+
βββ bomberman_env.py # Top-level PettingZoo AECEnv; basic_env() factory lives here
|
| 235 |
+
βββ dynamics.py # All game logic: phases, rewards, registry access
|
| 236 |
+
βββ arena.py # Maze generation (structural + destructible walls, tile placement)
|
| 237 |
+
βββ config.py # OmegaConf-based config tree; load_config() / default_config()
|
| 238 |
+
βββ observation.py # Build per-agent viewcone tensors; ViewChannel enum (25 channels)
|
| 239 |
+
βββ actions.py # ActionV2 enum, ActionMask (legal-move computation)
|
| 240 |
+
βββ types.py # Direction, Tile enums and shared type aliases
|
| 241 |
+
βββ renderer.py # Pygame renderer; HUD, replay recording
|
| 242 |
+
βββ flatten_dict.py # FlattenDictWrapper β flattens obs dict to 1-D array
|
| 243 |
+
βββ helpers.py # Miscellaneous utilities
|
| 244 |
+
βββ play.py # Interactive harness: launch with `uv run python -m til_environment.play`
|
| 245 |
+
β
|
| 246 |
+
βββ entities/
|
| 247 |
+
β βββ base.py # Entity base class (entity_id, team, position, alive)
|
| 248 |
+
β βββ dynamic.py # Agent, Base, Bomb classes
|
| 249 |
+
β βββ static.py # Mission, Recon, Resource, DestructibleWall, AttackIntent
|
| 250 |
+
β βββ protocols.py # Structural interfaces: Attacker, Defender, Vision, Item, Timed, ExternalSideEffect
|
| 251 |
+
β βοΏ½οΏ½β geometry.py # Blast/vision geometry helpers (RadiusAttack, SquareVision)
|
| 252 |
+
β βββ registry.py # EntityRegistry β indexed container for all live entities
|
| 253 |
+
β
|
| 254 |
+
βββ events/
|
| 255 |
+
βββ base.py # EmitRule dataclass + wrap_multi() β hooks entity methods to reward events
|
| 256 |
+
βββ rewards.py # Rewards accumulator (step + episode); award() method
|
| 257 |
+
|
| 258 |
+
```
|
| 259 |
+
|
| 260 |
+
### Key concepts
|
| 261 |
+
|
| 262 |
+
**Agents and teams.** There is exactly one agent per team. Agent IDs are `"agent_0"`, `"agent_1"`, β¦ `"agent_{N-1}"`. Each team also has one base.
|
| 263 |
+
|
| 264 |
+
**Death and respawn.** When an agent's HP reaches 0 it enters a *frozen* state for 3 steps β it stays on the grid but its entire action mask is zeroed. After the freeze it respawns at the same tile at full HP. Agents are never removed from the game; only step truncation ends an episode.
|
| 265 |
+
|
| 266 |
+
**Resource economy.** Each team has a shared resource pool (`team_resources`) and bomb stockpile (`team_bombs`). Resources accumulate passively every step at a fixed rate (`0.1`), and are also collected by stepping on resource tiles (`0.5`). Whenever `team_resources β₯ 1.5` the surplus auto-converts into bombs.
|
| 267 |
+
|
| 268 |
+
**Bombs.** A placed bomb counts down for `timer` steps then detonates in a cross-shaped blast of square radius `2` (so 2 + 1 + 2, where 1 is the center where the bomb is place). The blast damages agents, bases, and destroys destructible walls in range. Indestructible walls block the blast.
|
| 269 |
+
|
| 270 |
+
**Observation channels.** The viewcone tensor has 25 channels encoding visibility, wall edges (structural and destructible, per direction), tile types, ally/enemy entities, bomb presence and timers, and HP levels. Once more, see [`observation.py`](til-26-ae/til_environment/observation.py) (`ViewChannel` enum) for the full list.
|
til-26-ae-repo/til-26-ae/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
MIT License
|
| 2 |
+
|
| 3 |
+
Copyright (c) 2025 BrainHack TIL-AI
|
| 4 |
+
|
| 5 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 6 |
+
of this software and associated documentation files (the "Software"), to deal
|
| 7 |
+
in the Software without restriction, including without limitation the rights
|
| 8 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 9 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 10 |
+
furnished to do so, subject to the following conditions:
|
| 11 |
+
|
| 12 |
+
The above copyright notice and this permission notice shall be included in all
|
| 13 |
+
copies or substantial portions of the Software.
|
| 14 |
+
|
| 15 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 16 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 17 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 18 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 19 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 20 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 21 |
+
SOFTWARE.
|
til-26-ae-repo/til-26-ae/README.md
ADDED
|
@@ -0,0 +1,175 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# TIL Bomberman β Environment
|
| 2 |
+
|
| 3 |
+
## Quick Start
|
| 4 |
+
|
| 5 |
+
`play.py` controls: `W/A/S/D` or arrow keys to move, `B/F` to place a bomb, `SPACE` to stay, `T` to toggle the respawn-timer overlay, `R` to reset, `Q`/`ESC` to quit.
|
| 6 |
+
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
## Overview
|
| 10 |
+
|
| 11 |
+
A two-team [PettingZoo](https://pettingzoo.farama.org/) AEC environment built around bomb placement, base defence, and resource management. Each team controls one agent and one stationary base. Agents explore a maze-like arena, collect resource tokens to fund bombs, and destroy the opposing base. Agents are never removed from the game β instead, taking lethal damage freezes them in place for a configurable number of ticks before they respawn at full health on the same tile.
|
| 12 |
+
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
## Overall Structure
|
| 16 |
+
|
| 17 |
+
```
|
| 18 |
+
til_environment/
|
| 19 |
+
βββ bomberman_env.py β PettingZoo AECEnv wrapper (public API)
|
| 20 |
+
βββ dynamics.py β all mutable game state and phase logic
|
| 21 |
+
βββ arena.py β maze generation, tile placement, spawn assignment
|
| 22 |
+
βββ observation.py β multi-channel viewcone builders
|
| 23 |
+
βββ actions.py β Action enum, ActionMask builder
|
| 24 |
+
βββ entities/
|
| 25 |
+
β βββ dynamic.py β Agent, Base, Bomb (mutable entities)
|
| 26 |
+
β βββ static.py β Mission, Recon, Resource (collectible entities)
|
| 27 |
+
β βββ protocols.py β structural trait protocols (Defender, Attacker, β¦)
|
| 28 |
+
β βββ geometry.py β RadiusAttack blast-cell computation
|
| 29 |
+
β βββ registry.py β EntityRegistry (typed, indexed entity container)
|
| 30 |
+
βββ renderer.py β pygame rendering; never mutates game state
|
| 31 |
+
βββ config.py β OmegaConf dataclass config hierarchy
|
| 32 |
+
βββ bomberman_config.yaml β default configuration
|
| 33 |
+
βββ helpers.py β shared grid utilities
|
| 34 |
+
βββ types.py β Direction, Tile, Wall enums
|
| 35 |
+
```
|
| 36 |
+
|
| 37 |
+
**`bomberman_env.py`** is the entry point for external code. It subclasses `AECEnv`, delegates all logic to `Dynamics`, and exposes the standard `reset / step / observe / render` contract. `basic_env()` and `parallel_basic_env()` are convenience factory functions that wrap the raw env with `FlattenDictWrapper` and `frame_stack_v2`.
|
| 38 |
+
|
| 39 |
+
**`dynamics.py`** owns the mutable world state: agent positions and health, team resource and bomb counts, bomb timers, tile respawn queues, and the per-round phase pipeline. It composes `ArenaGenerator`, `ArenaState`, `EntityRegistry`, `ActionMask`, and the `Rewards` system.
|
| 40 |
+
|
| 41 |
+
**`arena.py`** generates the maze (Recursive Backtracking via `mazelib`), marks a fraction of walls as destructible, scatters mission/recon/resource tiles with rotational symmetry across teams, and assigns agent/base starting positions. The wall layout is cached between episodes if the seed does not change. A Perlin-noise + centre-gradient respawn map is generated each episode, giving shorter respawn delays near the map centre.
|
| 42 |
+
|
| 43 |
+
**`entities/`** contains the entity hierarchy. All entities share a common `Entity` base (unique ID, position, team). Behaviour is declared via structural protocols (`Defender`, `Attacker`, `Vision`, `Item`, `Timed`) rather than deep inheritance, so `Dynamics` can dispatch on traits without checking concrete types. `EntityRegistry` is an indexed `dict[str, Entity]` with typed accessors (`agents(team)`, `bases(team)`, `bombs()`, etc.).
|
| 44 |
+
|
| 45 |
+
**`observation.py`** builds the multi-channel float32 viewcone tensors that agents receive. Line-of-sight is computed with a vectorised supercover algorithm (numpy batch array indexing; precomputed per viewcone shape via `lru_cache`).
|
| 46 |
+
|
| 47 |
+
**`renderer.py`** is a stateless pygame drawer. In `"human"` mode it draws to a window; in `"rgb_array"` mode it returns a numpy array. It never reads or writes game state β it only reads the `ArenaState`, `EntityRegistry`, and the current `observations` dict passed to it.
|
| 48 |
+
|
| 49 |
+
**`config.py`** defines the full config hierarchy as OmegaConf dataclasses. Every tuneable parameter lives here; no scattered constants anywhere else.
|
| 50 |
+
|
| 51 |
+
---
|
| 52 |
+
|
| 53 |
+
## Action Space
|
| 54 |
+
|
| 55 |
+
Each agent has a flat `Discrete(6)` action space. Bases do not act.
|
| 56 |
+
|
| 57 |
+
| Value | Name | Effect |
|
| 58 |
+
|-------|-------------|------------------------------------------------------|
|
| 59 |
+
| 0 | FORWARD | Move one tile in the agent's current facing direction |
|
| 60 |
+
| 1 | BACKWARD | Move one tile opposite to facing |
|
| 61 |
+
| 2 | LEFT | Turn left (rotate facing 90Β°), no movement |
|
| 62 |
+
| 3 | RIGHT | Turn right (rotate facing 90Β°), no movement |
|
| 63 |
+
| 4 | STAY | No action |
|
| 64 |
+
| 5 | PLACE_BOMB | Place a bomb on the agent's current tile |
|
| 65 |
+
|
| 66 |
+
An **action mask** (shape `(6,)`, `uint8`) is included in every observation. Illegal actions are replaced with `STAY` and penalised. Movement is masked when a wall blocks the path (structural or destructible). `PLACE_BOMB` is masked when the team has no bombs available or the agent is frozen.
|
| 67 |
+
|
| 68 |
+
---
|
| 69 |
+
|
| 70 |
+
## Observation Space
|
| 71 |
+
|
| 72 |
+
Each agent receives a `Dict` observation:
|
| 73 |
+
|
| 74 |
+
| Key | Space | Description |
|
| 75 |
+
|------------------|-------------------------------------------|--------------------------------------------------|
|
| 76 |
+
| `agent_viewcone` | `Box(0,1, (H,W,25), float32)` | Multi-channel viewcone centred on the agent |
|
| 77 |
+
| `base_viewcone` | `Box(0,1, (B,B,25), float32)` | Square view centred on the team base |
|
| 78 |
+
| `direction` | `Discrete(4)` | Agent's current facing direction |
|
| 79 |
+
| `location` | `Box(0,G, (2,), uint8)` | Agent grid position |
|
| 80 |
+
| `base_location` | `Box(0,G, (2,), uint8)` | Team base grid position |
|
| 81 |
+
| `health` | `Box(0, max_health, (1,), float32)` | Agent HP |
|
| 82 |
+
| `frozen_ticks` | `Discrete(freeze_turns+1)` | Ticks remaining until respawn (0 = active) |
|
| 83 |
+
| `base_health` | `Box(0, max_health, (1,), float32)` | Team base HP |
|
| 84 |
+
| `team_resources` | `Box(0, max_resources, (1,), float32)` | Shared resource pool (bombs cost 1.0 unit each) |
|
| 85 |
+
| `team_bombs` | `Discrete(max_bombs+1)` | Bombs currently available to the team |
|
| 86 |
+
| `step` | `Discrete(num_iters+1)` | Current step index |
|
| 87 |
+
| `action_mask` | `Box(0,1, (6,), uint8)` | Legal actions this step |
|
| 88 |
+
|
| 89 |
+
### Viewcone channels (25 total)
|
| 90 |
+
|
| 91 |
+
The viewcone is oriented relative to the agent's facing direction. Channel 0 is a line-of-sight flag; channels 1β4 encode wall edges; channels 5β12 encode tile types and entity presence; channels 13β16 flag destructible wall edges; channels 17β24 encode bomb state and entity health ratios. Index by `ViewChannel.*` enum rather than raw integers.
|
| 92 |
+
|
| 93 |
+
---
|
| 94 |
+
|
| 95 |
+
## Game Loop and Phases
|
| 96 |
+
|
| 97 |
+
The environment follows the PettingZoo AEC protocol: agents act one at a time. Once all agents have submitted an action the round executes in a fixed phase order:
|
| 98 |
+
|
| 99 |
+
1. **Validate** β illegal actions are replaced with `STAY`; `invalid_action` penalty awarded
|
| 100 |
+
2. **Place bombs** β `PLACE_BOMB` actions spawn bombs; `team_bombs[team]` decremented
|
| 101 |
+
3. **Movement** β agents move or turn; collectibles are picked up inline
|
| 102 |
+
4. **Detonation** β expired bomb timers produce `AttackIntent` lists
|
| 103 |
+
5. **Damage** β intents applied to defenders; agents at 0 HP enter frozen state
|
| 104 |
+
6. **Upkeep** β bomb timers tick; resource pools accrue; auto-convert resources β bombs; frozen agents tick down and respawn
|
| 105 |
+
|
| 106 |
+
Termination fires when a team's base is destroyed. Truncation fires at `num_iters` steps. Frozen (dead) agents alone do **not** trigger termination.
|
| 107 |
+
|
| 108 |
+
---
|
| 109 |
+
|
| 110 |
+
## Resource and Bomb Economy
|
| 111 |
+
|
| 112 |
+
Each team maintains a floating-point resource pool (`team_resources`) and an integer bomb count (`team_bombs`). Resources accumulate every tick at `base_resource_rate` as long as the team's base is alive. Picking up a resource tile adds a fixed amount to the pool. When the pool reaches `bomb_cost` (default 1.5), it is decremented and `team_bombs` incremented. Bombs start the episode at `starting_bombs` (default 3).
|
| 113 |
+
|
| 114 |
+
---
|
| 115 |
+
|
| 116 |
+
## Configuration
|
| 117 |
+
|
| 118 |
+
All parameters live in `bomberman_config.yaml` (loaded into an OmegaConf `BombermanConfig`). Key sections:
|
| 119 |
+
|
| 120 |
+
```yaml
|
| 121 |
+
env:
|
| 122 |
+
grid_size: 16
|
| 123 |
+
num_teams: 2
|
| 124 |
+
num_iters: 200
|
| 125 |
+
dynamics:
|
| 126 |
+
vision:
|
| 127 |
+
left: 2
|
| 128 |
+
right: 2
|
| 129 |
+
ahead: 4
|
| 130 |
+
behind: 1
|
| 131 |
+
entities:
|
| 132 |
+
agent:
|
| 133 |
+
max_health: 100.0
|
| 134 |
+
freeze_turns: 3
|
| 135 |
+
base:
|
| 136 |
+
max_health: 200.0
|
| 137 |
+
bomb:
|
| 138 |
+
timer: 5
|
| 139 |
+
blast_radius: 2
|
| 140 |
+
resources:
|
| 141 |
+
bomb_cost: 1.5
|
| 142 |
+
base_resource_rate: 0.1
|
| 143 |
+
starting_bombs: 3
|
| 144 |
+
rewards:
|
| 145 |
+
collect_mission: 5.0
|
| 146 |
+
attack_kill: 10.0
|
| 147 |
+
destroy_enemy_base: 50.0
|
| 148 |
+
```
|
| 149 |
+
|
| 150 |
+
Load a custom config:
|
| 151 |
+
|
| 152 |
+
```python
|
| 153 |
+
from til_environment.config import load_config
|
| 154 |
+
from til_environment.bomberman_env import Bomberman
|
| 155 |
+
|
| 156 |
+
env = Bomberman(load_config("my_config.yaml"))
|
| 157 |
+
```
|
| 158 |
+
|
| 159 |
+
Or use the default:
|
| 160 |
+
|
| 161 |
+
```python
|
| 162 |
+
from til_environment.bomberman_env import basic_env
|
| 163 |
+
|
| 164 |
+
env = basic_env() # includes FlattenDictWrapper + frame_stack_v2
|
| 165 |
+
```
|
| 166 |
+
|
| 167 |
+
---
|
| 168 |
+
|
| 169 |
+
## Tests
|
| 170 |
+
|
| 171 |
+
```bash
|
| 172 |
+
uv run pytest til_environment/tests/ -v
|
| 173 |
+
```
|
| 174 |
+
|
| 175 |
+
The test suite covers bomb placement and detonation, destructible wall behaviour, resource/bomb economy, agent freeze and respawn, termination conditions, action masking, and the Perlin respawn map.
|