Model card: document the periodic checkpoints

README.md

@@ -1,168 +1,169 @@
----
-license: mit
-library_name: pytorch
-tags:
-  - reinforcement-learning
-  - multi-agent-reinforcement-learning
-  - deep-reinforcement-learning
-  - dqn
-  - dueling-dqn
-  - multi-agent-systems
-  - drone-swarm
-  - fault-tolerance
-  - domain-randomization
----
-
-# Multi-Drone Cooperative Search with Random Faults — trained policy
-
-Trained weights for the Multi-Agent Systems course project (MSc Artificial
-Intelligence, University of Bologna, A.Y. 2025/26).
-
-A team of `N` drones searches a 32×32 occupancy grid for one hidden target under
-partial observability, range-limited communication and a fixed time budget,
-while **any drone can permanently break at any moment**, with no central
-authority to announce the loss. A single Dueling Double DQN with parameter
-sharing drives every drone; per-episode domain randomization plus a compact
-context vector make that one policy a generalist across team size, sensing,
-communication range, obstacle density and fault rate.
-
-- **Code:** https://github.com/simoswish02/Multi-Agent-Systems-Project
-- **Report (49 pp.):** [`report/main.pdf`](https://github.com/simoswish02/Multi-Agent-Systems-Project/blob/main/report/main.pdf)
-- **Video walkthrough (~4 min):** https://youtu.be/az1LtAcZYZo
-
-## Files
-
-| File | Description |
-|---|---|
-| `mas_50k_dr_faults_ep50000.pt` | Final checkpoint of the 50,000-episode run. **Every number in Section 7 of the report comes from this file.** |
-
-## Usage
-
-```bash
-git clone https://github.com/simoswish02/Multi-Agent-Systems-Project
-cd Multi-Agent-Systems-Project
-conda env create -f environment.yaml && conda activate rl-drone
-
-# download the checkpoint into checkpoints/
-huggingface-cli download simoswish/MAS_MultiDroneExploration \
-    mas_50k_dr_faults_ep50000.pt --local-dir checkpoints/
-
-# watch it fly
-python main.py --mode eval --checkpoint checkpoints/mas_50k_dr_faults_ep50000.pt \
-               --fault-prob 0.003
-
-# reproduce the experiments of Section 7
-python testing/evaluate_policy.py --axis all --seeds 500 \
-       --checkpoint checkpoints/mas_50k_dr_faults_ep50000.pt
-```
-
-Loading it directly:
-
-```python
-import torch
-ckpt = torch.load("mas_50k_dr_faults_ep50000.pt", map_location="cpu")
-state_dict = ckpt["q_net"]          # 22.4M parameters
-# the file also carries optimizer / scheduler / epsilon state, so training
-# can be resumed with `python main.py --mode train --resume <file>`
-```
-
-## Model
-
-**Architecture** (`agents/networks.py`, report Section 4.4):
-
-- **Global trunk** over six 32×32 belief planes (Visited, Obstacle, Trajectory,
-  Target, Own\_Position, Broken): a ConvNeXt-style CNN, stages of width
-  96 → 192 → 384 at resolutions 32 → 16 → 8, with per-stage FiLM conditioning,
-  followed by a 3-layer / 8-head Transformer over the 8×8 = 64 spatial tokens
-  plus a CLS token and a context token.
-- **Local trunk** over a 13×13 egocentric crop with five channels: three
-  stride-1 ConvNeXt blocks at width 96 (the middle one dilated) → 384.
-- **Fusion and head:** concat (1024 + 384 + 5) → FC 1536 → 768 → 384 → dueling
-  head, `Q = V + A − mean(A)`, over 4 actions (up / down / left / right).
-- 22.4M parameters, float32.
-
-**Context vector** (`CTX_DIM = 5`), rebuilt every round for every drone:
-`[vision_radius, comm_range, n_agents, agent_id, n_alive_belief]`, each
-normalized by the domain-randomization maxima (6, 12, 4, 4, 4). `agent_id`
-breaks the parameter-sharing symmetry so co-located drones split up from step 0;
-`n_alive_belief` is that drone's *own* belief about how many teammates remain —
-decentralized, and possibly stale.
-
-## Training
-
-Independent / parameter-sharing Dueling Double DQN, one shared network, single
-phase of 50,000 episodes; Adam, weight decay 1e-6, gradient clip 1.0, γ = 0.97,
-3-step returns, Huber loss, uniform replay of 2·10⁵ transitions, batch 16, one
-gradient step every 4 agent-turns, hard target sync every 500 steps. ε decays
-1.0 → 0.05 (×0.9995/episode, floor at episode 5,990); the learning rate warms up
-over the first 10% of episodes to 5·10⁻⁴, then anneals cosine-wise to 10⁻⁶.
-
-Per-episode domain randomization: vision radius ∈ [1, 6], communication range
-∈ [2, 12], team size ∈ [1, 4], obstacle density ∈ [0.10, 0.30], per-turn fault
-probability ∈ [0, 0.003]. The grid size (32) is the one constant.
-
-## Results
-
-At the reference operating point (N = 3, r_vis = 3, r_comm = 5, ρ = 0.2, T = 200,
-no faults), 500 seeded instances, greedy:
-
-| | |
-|---|---|
-| Success rate | **87.6%** (Wilson CI [84.4, 90.2]) |
-| Map coverage | 53.7% |
-| SPL | 0.541 |
-
-Generalization (success %, endpoints of each sweep):
-
-| Axis | In-distribution | OOD probe |
-|---|---|---|
-| Team size N | 71.0 (N=1) → 89.2 (N=4) | 89.6, 90.4 (N = 5, 6) |
-| Vision r_vis | 70.6 → 87.2 | 87.0, 85.6 (r = 7, 8) |
-| Comm. r_comm | 87.4 → 89.0 | 87.6, 88.0, 89.6 (r = 0, 14, 16) |
-| Density ρ | 96.2 (0.10) → 53.4 (0.30) | 99.0, 34.6, 24.2 (ρ = 0, 0.35, 0.40) |
-
-Fault robustness — the project's headline result:
-
-| p_fault | 0 | 0.001 | 0.002 | 0.003 | 0.005 | 0.010 |
-|---|---|---|---|---|---|---|
-| Success [%] | 87.6 | 86.0 | 84.4 | 82.0 | 77.2 | 66.2 |
-| Drones lost (mean) | 0.00 | 0.22 | 0.42 | 0.58 | 0.91 | 1.51 |
-
-Across the entire trained fault range success falls by under six points while
-the team loses more than half a drone on average; at more than triple the
-trained maximum it still completes two missions in three, with no cliff.
-
-## Limitations
-
-- Simulation only: 4-connected grid moves, noiseless sensing, instantaneous
-  range-limited fusion. No sim-to-real transfer is claimed or tested.
-- Faults are i.i.d. across drones and turns and produce inert, non-blocking
-  wrecks. Correlated failures, passage-blocking wrecks and an adversary
-  choosing whom to disable are outside the model.
-- Obstacle density is the one axis whose extrapolation degrades, though a
-  breadth-first check of the evaluation instances shows most of that to be the
-  instance distribution becoming unsolvable rather than the policy failing
-  (Section 7.3 of the report).
-- Statistical, not formal, evidence: confidence intervals bound sampling error,
-  not worst-case behaviour on an unseen instance.
-
-Section 8 of the report discusses the dual-use profile of the capability, its
-treatment under the GDPR and the EU AI Act, and why this specific system would
-be unacceptable as an autonomous weapon.
-
-## Citation
-
-```bibtex
-@misc{rimondi2026multidrone,
-  author = {Rimondi, Simone},
-  title  = {Multi-Drone Cooperative Search with Random Faults},
-  year   = {2026},
-  note   = {Multi-Agent Systems project, MSc Artificial Intelligence,
-            University of Bologna},
-  url    = {https://github.com/simoswish02/Multi-Agent-Systems-Project}
-}
-```
-
-## License
-
-MIT.
+---
+license: mit
+library_name: pytorch
+tags:
+  - reinforcement-learning
+  - multi-agent-reinforcement-learning
+  - deep-reinforcement-learning
+  - dqn
+  - dueling-dqn
+  - multi-agent-systems
+  - drone-swarm
+  - fault-tolerance
+  - domain-randomization
+---
+
+# Multi-Drone Cooperative Search with Random Faults — trained policy
+
+Trained weights for the Multi-Agent Systems course project (MSc Artificial
+Intelligence, University of Bologna, A.Y. 2025/26).
+
+A team of `N` drones searches a 32×32 occupancy grid for one hidden target under
+partial observability, range-limited communication and a fixed time budget,
+while **any drone can permanently break at any moment**, with no central
+authority to announce the loss. A single Dueling Double DQN with parameter
+sharing drives every drone; per-episode domain randomization plus a compact
+context vector make that one policy a generalist across team size, sensing,
+communication range, obstacle density and fault rate.
+
+- **Code:** https://github.com/simoswish02/Multi-Agent-Systems-Project
+- **Report (49 pp.):** [`report/main.pdf`](https://github.com/simoswish02/Multi-Agent-Systems-Project/blob/main/report/main.pdf)
+- **Video walkthrough (~4 min):** https://youtu.be/az1LtAcZYZo
+
+## Files
+
+| File | Description |
+|---|---|
+| `mas_50k_dr_faults_ep50000.pt` | Final checkpoint of the 50,000-episode run. **Every number in Section 7 of the report comes from this file.** This is the one to download for normal use. |
+| `checkpoints/mas_50k_dr_faults_ep*.pt` | The 20 periodic snapshots taken every 2,500 episodes, including a copy of the final one. Needed only to reproduce the learning curve (`--axis epoch`), which re-evaluates each of them; ~5.1 GB in total. |
+
+## Usage
+
+```bash
+git clone https://github.com/simoswish02/Multi-Agent-Systems-Project
+cd Multi-Agent-Systems-Project
+conda env create -f environment.yaml && conda activate rl-drone
+
+# download the checkpoint into checkpoints/
+huggingface-cli download simoswish/MAS_MultiDroneExploration \
+    mas_50k_dr_faults_ep50000.pt --local-dir checkpoints/
+
+# watch it fly
+python main.py --mode eval --checkpoint checkpoints/mas_50k_dr_faults_ep50000.pt \
+               --fault-prob 0.003
+
+# reproduce the experiments of Section 7
+python testing/evaluate_policy.py --axis all --seeds 500 \
+       --checkpoint checkpoints/mas_50k_dr_faults_ep50000.pt
+```
+
+Loading it directly:
+
+```python
+import torch
+ckpt = torch.load("mas_50k_dr_faults_ep50000.pt", map_location="cpu")
+state_dict = ckpt["q_net"]          # 22.4M parameters
+# the file also carries optimizer / scheduler / epsilon state, so training
+# can be resumed with `python main.py --mode train --resume <file>`
+```
+
+## Model
+
+**Architecture** (`agents/networks.py`, report Section 4.4):
+
+- **Global trunk** over six 32×32 belief planes (Visited, Obstacle, Trajectory,
+  Target, Own\_Position, Broken): a ConvNeXt-style CNN, stages of width
+  96 → 192 → 384 at resolutions 32 → 16 → 8, with per-stage FiLM conditioning,
+  followed by a 3-layer / 8-head Transformer over the 8×8 = 64 spatial tokens
+  plus a CLS token and a context token.
+- **Local trunk** over a 13×13 egocentric crop with five channels: three
+  stride-1 ConvNeXt blocks at width 96 (the middle one dilated) → 384.
+- **Fusion and head:** concat (1024 + 384 + 5) → FC 1536 → 768 → 384 → dueling
+  head, `Q = V + A − mean(A)`, over 4 actions (up / down / left / right).
+- 22.4M parameters, float32.
+
+**Context vector** (`CTX_DIM = 5`), rebuilt every round for every drone:
+`[vision_radius, comm_range, n_agents, agent_id, n_alive_belief]`, each
+normalized by the domain-randomization maxima (6, 12, 4, 4, 4). `agent_id`
+breaks the parameter-sharing symmetry so co-located drones split up from step 0;
+`n_alive_belief` is that drone's *own* belief about how many teammates remain —
+decentralized, and possibly stale.
+
+## Training
+
+Independent / parameter-sharing Dueling Double DQN, one shared network, single
+phase of 50,000 episodes; Adam, weight decay 1e-6, gradient clip 1.0, γ = 0.97,
+3-step returns, Huber loss, uniform replay of 2·10⁵ transitions, batch 16, one
+gradient step every 4 agent-turns, hard target sync every 500 steps. ε decays
+1.0 → 0.05 (×0.9995/episode, floor at episode 5,990); the learning rate warms up
+over the first 10% of episodes to 5·10⁻⁴, then anneals cosine-wise to 10⁻⁶.
+
+Per-episode domain randomization: vision radius ∈ [1, 6], communication range
+∈ [2, 12], team size ∈ [1, 4], obstacle density ∈ [0.10, 0.30], per-turn fault
+probability ∈ [0, 0.003]. The grid size (32) is the one constant.
+
+## Results
+
+At the reference operating point (N = 3, r_vis = 3, r_comm = 5, ρ = 0.2, T = 200,
+no faults), 500 seeded instances, greedy:
+
+| | |
+|---|---|
+| Success rate | **87.6%** (Wilson CI [84.4, 90.2]) |
+| Map coverage | 53.7% |
+| SPL | 0.541 |
+
+Generalization (success %, endpoints of each sweep):
+
+| Axis | In-distribution | OOD probe |
+|---|---|---|
+| Team size N | 71.0 (N=1) → 89.2 (N=4) | 89.6, 90.4 (N = 5, 6) |
+| Vision r_vis | 70.6 → 87.2 | 87.0, 85.6 (r = 7, 8) |
+| Comm. r_comm | 87.4 → 89.0 | 87.6, 88.0, 89.6 (r = 0, 14, 16) |
+| Density ρ | 96.2 (0.10) → 53.4 (0.30) | 99.0, 34.6, 24.2 (ρ = 0, 0.35, 0.40) |
+
+Fault robustness — the project's headline result:
+
+| p_fault | 0 | 0.001 | 0.002 | 0.003 | 0.005 | 0.010 |
+|---|---|---|---|---|---|---|
+| Success [%] | 87.6 | 86.0 | 84.4 | 82.0 | 77.2 | 66.2 |
+| Drones lost (mean) | 0.00 | 0.22 | 0.42 | 0.58 | 0.91 | 1.51 |
+
+Across the entire trained fault range success falls by under six points while
+the team loses more than half a drone on average; at more than triple the
+trained maximum it still completes two missions in three, with no cliff.
+
+## Limitations
+
+- Simulation only: 4-connected grid moves, noiseless sensing, instantaneous
+  range-limited fusion. No sim-to-real transfer is claimed or tested.
+- Faults are i.i.d. across drones and turns and produce inert, non-blocking
+  wrecks. Correlated failures, passage-blocking wrecks and an adversary
+  choosing whom to disable are outside the model.
+- Obstacle density is the one axis whose extrapolation degrades, though a
+  breadth-first check of the evaluation instances shows most of that to be the
+  instance distribution becoming unsolvable rather than the policy failing
+  (Section 7.3 of the report).
+- Statistical, not formal, evidence: confidence intervals bound sampling error,
+  not worst-case behaviour on an unseen instance.
+
+Section 8 of the report discusses the dual-use profile of the capability, its
+treatment under the GDPR and the EU AI Act, and why this specific system would
+be unacceptable as an autonomous weapon.
+
+## Citation
+
+```bibtex
+@misc{rimondi2026multidrone,
+  author = {Rimondi, Simone},
+  title  = {Multi-Drone Cooperative Search with Random Faults},
+  year   = {2026},
+  note   = {Multi-Agent Systems project, MSc Artificial Intelligence,
+            University of Bologna},
+  url    = {https://github.com/simoswish02/Multi-Agent-Systems-Project}
+}
+```
+
+## License
+
+MIT.