simoswish commited on
Commit
c2c7ea6
Β·
verified Β·
1 Parent(s): dfede94

Model card: architecture, training setup, results and limitations

Browse files
Files changed (1) hide show
  1. README.md +165 -0
README.md CHANGED
@@ -1,3 +1,168 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ library_name: pytorch
4
+ tags:
5
+ - reinforcement-learning
6
+ - multi-agent-reinforcement-learning
7
+ - deep-reinforcement-learning
8
+ - dqn
9
+ - dueling-dqn
10
+ - multi-agent-systems
11
+ - drone-swarm
12
+ - fault-tolerance
13
+ - domain-randomization
14
  ---
15
+
16
+ # Multi-Drone Cooperative Search with Random Faults β€” trained policy
17
+
18
+ Trained weights for the Multi-Agent Systems course project (MSc Artificial
19
+ Intelligence, University of Bologna, A.Y. 2025/26).
20
+
21
+ A team of `N` drones searches a 32Γ—32 occupancy grid for one hidden target under
22
+ partial observability, range-limited communication and a fixed time budget,
23
+ while **any drone can permanently break at any moment**, with no central
24
+ authority to announce the loss. A single Dueling Double DQN with parameter
25
+ sharing drives every drone; per-episode domain randomization plus a compact
26
+ context vector make that one policy a generalist across team size, sensing,
27
+ communication range, obstacle density and fault rate.
28
+
29
+ - **Code:** https://github.com/simoswish02/Multi-Agent-Systems-Project
30
+ - **Report (49 pp.):** [`report/main.pdf`](https://github.com/simoswish02/Multi-Agent-Systems-Project/blob/main/report/main.pdf)
31
+ - **Video walkthrough (~4 min):** https://youtu.be/az1LtAcZYZo
32
+
33
+ ## Files
34
+
35
+ | File | Description |
36
+ |---|---|
37
+ | `mas_50k_dr_faults_ep50000.pt` | Final checkpoint of the 50,000-episode run. **Every number in Section 7 of the report comes from this file.** |
38
+
39
+ ## Usage
40
+
41
+ ```bash
42
+ git clone https://github.com/simoswish02/Multi-Agent-Systems-Project
43
+ cd Multi-Agent-Systems-Project
44
+ conda env create -f environment.yaml && conda activate rl-drone
45
+
46
+ # download the checkpoint into checkpoints/
47
+ huggingface-cli download simoswish/MAS_MultiDroneExploration \
48
+ mas_50k_dr_faults_ep50000.pt --local-dir checkpoints/
49
+
50
+ # watch it fly
51
+ python main.py --mode eval --checkpoint checkpoints/mas_50k_dr_faults_ep50000.pt \
52
+ --fault-prob 0.003
53
+
54
+ # reproduce the experiments of Section 7
55
+ python testing/evaluate_policy.py --axis all --seeds 500 \
56
+ --checkpoint checkpoints/mas_50k_dr_faults_ep50000.pt
57
+ ```
58
+
59
+ Loading it directly:
60
+
61
+ ```python
62
+ import torch
63
+ ckpt = torch.load("mas_50k_dr_faults_ep50000.pt", map_location="cpu")
64
+ state_dict = ckpt["q_net"] # 22.4M parameters
65
+ # the file also carries optimizer / scheduler / epsilon state, so training
66
+ # can be resumed with `python main.py --mode train --resume <file>`
67
+ ```
68
+
69
+ ## Model
70
+
71
+ **Architecture** (`agents/networks.py`, report Section 4.4):
72
+
73
+ - **Global trunk** over six 32Γ—32 belief planes (Visited, Obstacle, Trajectory,
74
+ Target, Own\_Position, Broken): a ConvNeXt-style CNN, stages of width
75
+ 96 β†’ 192 β†’ 384 at resolutions 32 β†’ 16 β†’ 8, with per-stage FiLM conditioning,
76
+ followed by a 3-layer / 8-head Transformer over the 8Γ—8 = 64 spatial tokens
77
+ plus a CLS token and a context token.
78
+ - **Local trunk** over a 13Γ—13 egocentric crop with five channels: three
79
+ stride-1 ConvNeXt blocks at width 96 (the middle one dilated) β†’ 384.
80
+ - **Fusion and head:** concat (1024 + 384 + 5) β†’ FC 1536 β†’ 768 β†’ 384 β†’ dueling
81
+ head, `Q = V + A βˆ’ mean(A)`, over 4 actions (up / down / left / right).
82
+ - 22.4M parameters, float32.
83
+
84
+ **Context vector** (`CTX_DIM = 5`), rebuilt every round for every drone:
85
+ `[vision_radius, comm_range, n_agents, agent_id, n_alive_belief]`, each
86
+ normalized by the domain-randomization maxima (6, 12, 4, 4, 4). `agent_id`
87
+ breaks the parameter-sharing symmetry so co-located drones split up from step 0;
88
+ `n_alive_belief` is that drone's *own* belief about how many teammates remain β€”
89
+ decentralized, and possibly stale.
90
+
91
+ ## Training
92
+
93
+ Independent / parameter-sharing Dueling Double DQN, one shared network, single
94
+ phase of 50,000 episodes; Adam, weight decay 1e-6, gradient clip 1.0, Ξ³ = 0.97,
95
+ 3-step returns, Huber loss, uniform replay of 2·10⁡ transitions, batch 16, one
96
+ gradient step every 4 agent-turns, hard target sync every 500 steps. Ξ΅ decays
97
+ 1.0 β†’ 0.05 (Γ—0.9995/episode, floor at episode 5,990); the learning rate warms up
98
+ over the first 10% of episodes to 5·10⁻⁴, then anneals cosine-wise to 10⁻⁢.
99
+
100
+ Per-episode domain randomization: vision radius ∈ [1, 6], communication range
101
+ ∈ [2, 12], team size ∈ [1, 4], obstacle density ∈ [0.10, 0.30], per-turn fault
102
+ probability ∈ [0, 0.003]. The grid size (32) is the one constant.
103
+
104
+ ## Results
105
+
106
+ At the reference operating point (N = 3, r_vis = 3, r_comm = 5, ρ = 0.2, T = 200,
107
+ no faults), 500 seeded instances, greedy:
108
+
109
+ | | |
110
+ |---|---|
111
+ | Success rate | **87.6%** (Wilson CI [84.4, 90.2]) |
112
+ | Map coverage | 53.7% |
113
+ | SPL | 0.541 |
114
+
115
+ Generalization (success %, endpoints of each sweep):
116
+
117
+ | Axis | In-distribution | OOD probe |
118
+ |---|---|---|
119
+ | Team size N | 71.0 (N=1) β†’ 89.2 (N=4) | 89.6, 90.4 (N = 5, 6) |
120
+ | Vision r_vis | 70.6 β†’ 87.2 | 87.0, 85.6 (r = 7, 8) |
121
+ | Comm. r_comm | 87.4 β†’ 89.0 | 87.6, 88.0, 89.6 (r = 0, 14, 16) |
122
+ | Density ρ | 96.2 (0.10) β†’ 53.4 (0.30) | 99.0, 34.6, 24.2 (ρ = 0, 0.35, 0.40) |
123
+
124
+ Fault robustness β€” the project's headline result:
125
+
126
+ | p_fault | 0 | 0.001 | 0.002 | 0.003 | 0.005 | 0.010 |
127
+ |---|---|---|---|---|---|---|
128
+ | Success [%] | 87.6 | 86.0 | 84.4 | 82.0 | 77.2 | 66.2 |
129
+ | Drones lost (mean) | 0.00 | 0.22 | 0.42 | 0.58 | 0.91 | 1.51 |
130
+
131
+ Across the entire trained fault range success falls by under six points while
132
+ the team loses more than half a drone on average; at more than triple the
133
+ trained maximum it still completes two missions in three, with no cliff.
134
+
135
+ ## Limitations
136
+
137
+ - Simulation only: 4-connected grid moves, noiseless sensing, instantaneous
138
+ range-limited fusion. No sim-to-real transfer is claimed or tested.
139
+ - Faults are i.i.d. across drones and turns and produce inert, non-blocking
140
+ wrecks. Correlated failures, passage-blocking wrecks and an adversary
141
+ choosing whom to disable are outside the model.
142
+ - Obstacle density is the one axis whose extrapolation degrades, though a
143
+ breadth-first check of the evaluation instances shows most of that to be the
144
+ instance distribution becoming unsolvable rather than the policy failing
145
+ (Section 7.3 of the report).
146
+ - Statistical, not formal, evidence: confidence intervals bound sampling error,
147
+ not worst-case behaviour on an unseen instance.
148
+
149
+ Section 8 of the report discusses the dual-use profile of the capability, its
150
+ treatment under the GDPR and the EU AI Act, and why this specific system would
151
+ be unacceptable as an autonomous weapon.
152
+
153
+ ## Citation
154
+
155
+ ```bibtex
156
+ @misc{rimondi2026multidrone,
157
+ author = {Rimondi, Simone},
158
+ title = {Multi-Drone Cooperative Search with Random Faults},
159
+ year = {2026},
160
+ note = {Multi-Agent Systems project, MSc Artificial Intelligence,
161
+ University of Bologna},
162
+ url = {https://github.com/simoswish02/Multi-Agent-Systems-Project}
163
+ }
164
+ ```
165
+
166
+ ## License
167
+
168
+ MIT.