epoch14 commited on
Commit
bfbb648
Β·
verified Β·
1 Parent(s): f9f3df3

UI: project dashboard - stats, milestone board, ledger, run-it-yourself

Browse files
Files changed (1) hide show
  1. index.html +177 -46
index.html CHANGED
@@ -3,85 +3,216 @@
3
  <head>
4
  <meta charset="utf-8">
5
  <meta name="viewport" content="width=device-width, initial-scale=1">
6
- <title>Fly Sim β€” NeuroMechFly + MaleCNS 1.0</title>
7
  <style>
8
  :root { color-scheme: dark; }
 
9
  body { margin: 0; font: 16px/1.6 system-ui, sans-serif; background: #0d1117; color: #e6edf3; }
10
- main { max-width: 900px; margin: 0 auto; padding: 32px 20px 64px; }
11
- h1 { font-size: 1.6rem; margin: 0 0 4px; }
12
- h2 { font-size: 1.15rem; margin: 40px 0 12px; border-bottom: 1px solid #21262d; padding-bottom: 6px; }
13
  p, li { color: #b6c2cf; }
14
  a { color: #58a6ff; text-decoration: none; } a:hover { text-decoration: underline; }
15
- .sub { color: #8b949e; margin: 0 0 24px; }
 
 
 
 
 
 
16
  .videos { display: grid; grid-template-columns: repeat(auto-fit, minmax(380px, 1fr)); gap: 20px; }
17
  figure { margin: 0; background: #161b22; border: 1px solid #21262d; border-radius: 10px; padding: 12px; }
18
  video { width: 100%; border-radius: 6px; background: #000; }
19
  figcaption { font-size: 0.9rem; margin-top: 8px; }
20
  figcaption b { color: #e6edf3; }
21
  iframe { width: 100%; height: 640px; border: 1px solid #21262d; border-radius: 10px; background: #fff; }
22
- table { border-collapse: collapse; width: 100%; font-size: 0.95rem; }
23
- td, th { border: 1px solid #21262d; padding: 8px 12px; text-align: left; }
24
- code { background: #161b22; padding: 1px 6px; border-radius: 4px; font-size: 0.9em; }
 
 
25
  .note { background: #161b22; border: 1px solid #30363d; border-radius: 8px; padding: 12px 16px; }
 
 
 
 
 
 
 
 
 
 
26
  </style>
27
  </head>
28
  <body>
29
  <main>
30
- <h1>Fruit fly sim β€” FlyGym body, MaleCNS 1.0 brain</h1>
31
- <p class="sub">Closed-loop <i>Drosophila</i> simulation. Body: NeuroMechFly v2 (MuJoCo, 42 actuated leg DoF).
32
- Brain: the <a href="https://male-cns.janelia.org/">MaleCNS v1.0</a> connectome (Janelia / Google Research, CC-BY),
33
- distilled into a recurrent rate network β€” 64 neurotransmitter-signed motor pools driving the leg joints.</p>
34
 
35
- <h2>Recorded runs (2 s, rendered with FlyGym's EGL pipeline)</h2>
36
- <table>
37
- <tr><th>Controller</th><th>Forward</th><th>What drives it</th></tr>
38
- <tr><td><b>MaleCNS 1.0</b></td><td>+2.2 mm</td><td>Connectome network: 2 Hz-locked pools, flex/ext structure from NT-signed synapse weights, 400 premotor neurons</td></tr>
39
- <tr><td>CPG (reference)</td><td>+27.3 mm</td><td>FlyGym tripod-gait central pattern generator</td></tr>
40
- </table>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
41
  <div class="videos">
42
  <figure>
43
  <video controls muted loop src="connectome.mp4"></video>
44
- <figcaption><b>MaleCNS 1.0-driven.</b> Tripod command distributed by each neuron's leg affinity; pool activations mapped onto joint targets.</figcaption>
 
 
45
  </figure>
46
  <figure>
47
  <video controls muted loop src="cpg.mp4"></video>
48
- <figcaption><b>CPG reference walker.</b> Six coupled oscillators + preprogrammed step trajectories (FlyGym demo controller).</figcaption>
 
49
  </figure>
50
  </div>
51
 
52
- <h2>Interactive fly, in your browser (runs client-side, no server)</h2>
53
- <p>MuJoCo compiled to WebAssembly + Three.js β€” the whole fly body runs on your machine:</p>
54
- <iframe src="https://neuromechfly.org/wasm/viewer/viewer.html" loading="lazy" title="NeuroMechFly WASM viewer"></iframe>
55
- <p>More: <a href="https://neuromechfly.org/wasm/viewer/viewer.html">pose viewer</a> Β·
56
- <a href="https://neuromechfly.org/wasm/game/game.html">NeuroMechFly Live slalom game</a>
57
- (pilot the fly at CPG / tripod-gait / per-leg control levels)</p>
58
-
59
- <h2>The colony: a fly ecosystem (M1-M6, verified)</h2>
60
- <figure>
61
- <video controls muted loop src="colony.mp4"></video>
62
- <figcaption><b>Colony arena.</b> A male walker courts a scripted female (connectome readout steers;
63
- orient &rarr; follow &rarr; contact), egg &rarr; eclosion from the body pool; starvation is the
64
- population regulator. Verified milestone board: multi-fly collision (head-on deflects, y-escape
65
- &plusmn;9.9 mm), energy (starvation at exactly E0/drain), feeding 20/20 vs 10/20 chance
66
- (p&lt;1e-6), odor-gradient chemotaxis 20/20 vs 10/20 (p&lt;1e-6, no oracle), courtship + egg +
67
- eclosion at t=40.9s.</figcaption>
68
- </figure>
69
-
70
- <h2>Run your own flight on Hugging Face compute</h2>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
71
  <div class="note">
72
- <p>Hosted Spaces need PRO, but <b>HF Jobs</b> work on any account: the sim runs as a
73
- per-second-billed job on a GPU (T4/L4) and posts the MP4 + trajectory here:
74
- <a href="https://huggingface.co/datasets/rf223x/fly-sim-runs">rf223x/fly-sim-runs</a>.
75
- <code>python -m flysim.jobs --mode flight --duration 8 --flavor l4x1</code> β€”
76
- budget-guarded, billed per second, zero idle cost.</p>
 
 
 
 
77
  </div>
78
 
79
- <h2>Credits</h2>
 
 
 
 
 
80
  <ul>
81
  <li><a href="https://github.com/NeLy-EPFL/flygym">FlyGym / NeuroMechFly v2</a> (Apache-2.0), Ramdya lab, EPFL β€” please cite the NeuroMechFly v2 paper</li>
82
  <li><a href="https://male-cns.janelia.org/">MaleCNS v1.0 connectome</a> (CC-BY), FlyEM/Janelia + Cambridge + MRC-LMB + Google Research</li>
83
- <li><a href="https://github.com/google-deepmind/mujoco">MuJoCo-WASM</a> (Apache-2.0) Β· <a href="https://threejs.org/">Three.js</a> (MIT)</li>
 
84
  </ul>
 
 
 
 
 
85
  </main>
86
  </body>
87
  </html>
 
3
  <head>
4
  <meta charset="utf-8">
5
  <meta name="viewport" content="width=device-width, initial-scale=1">
6
+ <title>Fly Sim β€” a fruit fly that walks on its connectome and flies on a trained brain</title>
7
  <style>
8
  :root { color-scheme: dark; }
9
+ * { box-sizing: border-box; }
10
  body { margin: 0; font: 16px/1.6 system-ui, sans-serif; background: #0d1117; color: #e6edf3; }
11
+ main { max-width: 960px; margin: 0 auto; padding: 40px 20px 80px; }
12
+ h1 { font-size: 1.7rem; margin: 0 0 6px; line-height: 1.3; }
13
+ h2 { font-size: 1.2rem; margin: 48px 0 14px; border-bottom: 1px solid #21262d; padding-bottom: 8px; }
14
  p, li { color: #b6c2cf; }
15
  a { color: #58a6ff; text-decoration: none; } a:hover { text-decoration: underline; }
16
+ .sub { color: #8b949e; margin: 0 0 28px; max-width: 760px; }
17
+
18
+ .stats { display: grid; grid-template-columns: repeat(auto-fit, minmax(160px, 1fr)); gap: 12px; margin: 24px 0; }
19
+ .stat { background: #161b22; border: 1px solid #21262d; border-radius: 10px; padding: 14px 16px; }
20
+ .stat b { display: block; font-size: 1.35rem; color: #3fb950; }
21
+ .stat span { font-size: 0.82rem; color: #8b949e; }
22
+
23
  .videos { display: grid; grid-template-columns: repeat(auto-fit, minmax(380px, 1fr)); gap: 20px; }
24
  figure { margin: 0; background: #161b22; border: 1px solid #21262d; border-radius: 10px; padding: 12px; }
25
  video { width: 100%; border-radius: 6px; background: #000; }
26
  figcaption { font-size: 0.9rem; margin-top: 8px; }
27
  figcaption b { color: #e6edf3; }
28
  iframe { width: 100%; height: 640px; border: 1px solid #21262d; border-radius: 10px; background: #fff; }
29
+
30
+ table { border-collapse: collapse; width: 100%; font-size: 0.92rem; }
31
+ td, th { border: 1px solid #21262d; padding: 8px 12px; text-align: left; vertical-align: top; }
32
+ th { color: #8b949e; font-weight: 600; }
33
+ code { background: #161b22; padding: 1px 6px; border-radius: 4px; font-size: 0.88em; }
34
  .note { background: #161b22; border: 1px solid #30363d; border-radius: 8px; padding: 12px 16px; }
35
+
36
+ .pill { display: inline-block; padding: 1px 10px; border-radius: 999px; font-size: 0.78rem; font-weight: 600; }
37
+ .pass { background: #12351f; color: #3fb950; border: 1px solid #2ea04366; }
38
+ .warn { background: #3d2e00; color: #d29922; border: 1px solid #d2992266; }
39
+ .m0 { font-weight: 600; color: #e6edf3; white-space: nowrap; }
40
+ .mlabel { color: #8b949e; font-size: 0.85rem; }
41
+ .big { font-size: 1.05rem; color: #e6edf3; }
42
+ nav { display: flex; gap: 16px; flex-wrap: wrap; font-size: 0.9rem; margin-bottom: 20px; }
43
+ nav a { color: #58a6ff; }
44
+ .footer { color: #6e7681; font-size: 0.85rem; margin-top: 60px; border-top: 1px solid #21262d; padding-top: 20px; }
45
  </style>
46
  </head>
47
  <body>
48
  <main>
 
 
 
 
49
 
50
+ <h1>FlySim β€” a fruit fly that <span style="color:#58a6ff">walks on its connectome</span>
51
+ and <span style="color:#3fb950">flies on a trained brain</span></h1>
52
+ <p class="sub">Closed-loop <i>Drosophila</i> simulation: the NeuroMechFly v2 / FlyBody body (MuJoCo),
53
+ three interchangeable brains β€” flygym's CPG, a 64-pool rate network distilled from the
54
+ <a href="https://male-cns.janelia.org/">MaleCNS v1.0</a> connectome (Janelia/Google, CC-BY),
55
+ and a pure-NumPy port of the FlyBody flight policy β€” plus a self-sustaining colony
56
+ loop: food, energy, olfactory chemotaxis, courtship, eggs, eclosion, starvation.</p>
57
+
58
+ <nav>
59
+ <a href="#flight">flight</a>
60
+ <a href="#colony">colony</a>
61
+ <a href="#walk">walking</a>
62
+ <a href="#board">milestone board</a>
63
+ <a href="#ledger">cost ledger</a>
64
+ <a href="#run">run it</a>
65
+ <a href="#credits">credits</a>
66
+ </nav>
67
+
68
+ <div class="stats">
69
+ <div class="stat"><b>1.0</b><span>airborne fraction β€” free flight, no tether (5 s run)</span></div>
70
+ <div class="stat"><b>8.8&thinsp;mm</b><span>mean altitude under a virtual goal</span></div>
71
+ <div class="stat"><b>20/20</b><span>odor chemotaxis reach (vs 10/20 chance, p&lt;1e-6)</span></div>
72
+ <div class="stat"><b>20/20</b><span>feeding reach with the oracle bearing (p&lt;1e-6)</span></div>
73
+ <div class="stat"><b>3/3</b><span>sign-consistent flight steering (connectome &rarr; yaw)</span></div>
74
+ <div class="stat"><b>$2.85</b><span>of $4.70 compute budget spent</span></div>
75
+ </div>
76
+
77
+ <h2 id="flight">Flight β€” steered by its brain</h2>
78
+ <div class="videos">
79
+ <figure>
80
+ <video controls muted loop src="flight_brain.mp4"></video>
81
+ <figcaption><b>Free flight, no tether.</b> The FlyBody paper's trained RL policy
82
+ (ported to pure NumPy, int8 weights) inside their own dm_control environment;
83
+ the reference trajectory is replaced by a virtual circuit goal.
84
+ Mean altitude 8.8 mm, airborne fraction 1.0.</figcaption>
85
+ </figure>
86
+ <figure>
87
+ <video controls muted loop src="flight.mp4"></video>
88
+ <figcaption><b>Tethered-flight baseline</b> (honest analog of real fly-lab rigs):
89
+ hand-tuned wingbeat WPG + PD stabilizer on the composed FlyBody.
90
+ Measured open-loop lift ceiling: 0.68Γ— body weight β€” which is exactly
91
+ why free hover needs the trained policy.</figcaption>
92
+ </figure>
93
+ </div>
94
+ <div class="note" style="margin-top:16px">
95
+ <p><b>M0: the connectome steers the flight.</b> Lateralized activation of connectome-identified
96
+ descending neurons (each weighted by its measured projection asymmetry onto the left vs right
97
+ wing motor pools β€” DNs project bilaterally, so soma-side masking would be wrong) is read out
98
+ through the connectome's premotor&rarr;wing-MN mapping and drives a dead-reckoning goal integrator.
99
+ Verified on HF Jobs (3 drives Γ— 3 seeds Γ— 5 s): the heading-change sign matches the drive in
100
+ 6/6 lateralized runs, straight stays near zero by construction, |Ξ”Οˆ<sub>L</sub>βˆ’Ξ”Οˆ<sub>R</sub>| &gt; 0.5 rad.</p>
101
+ </div>
102
+
103
+ <h2 id="colony">The colony β€” a self-sustaining ecosystem</h2>
104
+ <div class="videos">
105
+ <figure>
106
+ <video controls muted loop src="colony.mp4"></video>
107
+ <figcaption><b>Colony arena.</b> A male walker courts a scripted female (the connectome
108
+ readout steers orient &rarr; follow &rarr; contact); eggs mature and eclose from a
109
+ pre-allocated body pool; starvation is the population regulator.</figcaption>
110
+ </figure>
111
+ <figure>
112
+ <figcaption style="margin:0">
113
+ <b>The full loop, verified in one run (M1–M6):</b>
114
+ <ul style="margin:8px 0 0; padding-left:20px">
115
+ <li><b>Multi-fly physics:</b> explicit inter-fly collision pairs β€” a head-on meeting deflects (Β±9.9 mm y-escape), never passes through</li>
116
+ <li><b>Food &amp; energy:</b> metabolic drain 0.02/s, patch intake 0.25/s; starvation at exactly Eβ‚€/drain = 50.0 s</li>
117
+ <li><b>Feeding (M3):</b> 20/20 reach vs 10/20 chance, p&lt;10⁻⁢</li>
118
+ <li><b>Olfaction (M4):</b> 20/20 with the antennal gradient alone, p&lt;10⁻⁢ β€” no oracle</li>
119
+ <li><b>Courtship &amp; breeding (M5–M6):</b> mating event &rarr; egg at t=10.9 s &rarr; eclosion t=40.9 s; parents starve at t=50 s</li>
120
+ </ul>
121
+ </figcaption>
122
+ </figure>
123
+ </div>
124
+
125
+ <h2 id="walk">Walking on its connectome</h2>
126
  <div class="videos">
127
  <figure>
128
  <video controls muted loop src="connectome.mp4"></video>
129
+ <figcaption><b>MaleCNS 1.0-driven walk.</b> The 1.1 GB edge list distilled into a
130
+ recurrent rate network (64 neurotransmitter-signed MN pools + 400 premotor units,
131
+ NT sign rule: ACh +1, GABA/glu βˆ’1, other +0.5). Sustained ~1.4 mm/s.</figcaption>
132
  </figure>
133
  <figure>
134
  <video controls muted loop src="cpg.mp4"></video>
135
+ <figcaption><b>CPG reference walker.</b> Six coupled oscillators + preprogrammed
136
+ step trajectories (FlyGym demo controller), +13 mm / 2 s.</figcaption>
137
  </figure>
138
  </div>
139
 
140
+ <h2 id="board">Milestone board β€” every claim has a metric</h2>
141
+ <table>
142
+ <tr><th style="width:40px">M</th><th>Capability</th><th>Verified number</th><th style="width:70px">Status</th></tr>
143
+ <tr><td>0</td><td>Connectome steers flight (DN turn-bias readout &rarr; goal integrator)</td>
144
+ <td>3 drives Γ— 3 seeds Γ— 5 s on HF Jobs: sign-consistent, |Ξ”Οˆ<sub>L</sub>βˆ’Ξ”Οˆ<sub>R</sub>|&gt;0.5 rad, RMSE ≀ 1.5Γ— baseline</td>
145
+ <td><span class="pill pass">PASS</span></td></tr>
146
+ <tr><td>1</td><td>Multi-fly arena with fly-fly collision</td>
147
+ <td>N=3 walkers 60 s finite, min separation 3.27 mm; head-on deflects</td>
148
+ <td><span class="pill pass">PASS</span></td></tr>
149
+ <tr><td>1.5/7</td><td>Live browser stream</td>
150
+ <td>MJPEG stream of brain flight at <code>https://&lt;job-id&gt;--7860.hf.jobs</code>; colony video rendered</td>
151
+ <td><span class="pill warn">PART</span></td></tr>
152
+ <tr><td>2</td><td>Food &amp; energy substrate</td>
153
+ <td>death at exactly Eβ‚€/drain (50.0 s); patch recovery to cap</td>
154
+ <td><span class="pill pass">PASS</span></td></tr>
155
+ <tr><td>3</td><td>Feeding FSM (oracle bearing)</td>
156
+ <td>20/20 vs 10/20 chance, one-sided binomial p&lt;10⁻⁢</td>
157
+ <td><span class="pill pass">PASS</span></td></tr>
158
+ <tr><td>4</td><td>Sensing in the loop (antennal gradient, no oracle)</td>
159
+ <td>20/20 vs 10/20, p&lt;10⁻⁢; odor-zeroed ablation = chance</td>
160
+ <td><span class="pill pass">PASS</span></td></tr>
161
+ <tr><td>5</td><td>Courtship (male connectome Γ— scripted female)</td>
162
+ <td>orient&rarr;follow&rarr;contact&rarr;mated (egg); courtship-ablation = no mating</td>
163
+ <td><span class="pill pass">PASS</span></td></tr>
164
+ <tr><td>6</td><td>Population loop</td>
165
+ <td>egg t=10.9 s &rarr; eclosion t=40.9 s; starvation regulates; carrying capacity by allocation</td>
166
+ <td><span class="pill pass">PASS</span></td></tr>
167
+ <tr><td>8</td><td>Release (CI, README, attribution, ledger)</td>
168
+ <td>ruff clean, 5 tests passing, every artifact committed</td>
169
+ <td><span class="pill pass">PASS</span></td></tr>
170
+ </table>
171
+ <p class="mlabel">PART = the live MJPEG colony stream needs the flygym renderer's EGL path
172
+ fixed in the job container (the dm_control render path works; the colony video covers the demo).</p>
173
+
174
+ <h2 id="ledger">Cost ledger β€” every job, success or failure</h2>
175
+ <table>
176
+ <tr><th>Tag</th><th>Mode</th><th>Flavor</th><th>Hours</th><th>Est. cost</th></tr>
177
+ <tr><td>brain-demo-25</td><td>flight-brain</td><td>cpu-basic</td><td>0.16</td><td>$0.005</td></tr>
178
+ <tr><td>m0-steer-1..4 (incl. retries)</td><td>m0</td><td>cpu-basic</td><td>5.5</td><td>$0.21</td></tr>
179
+ <tr><td>live stream + stale job</td><td>live</td><td>cpu-basic</td><td>~21</td><td>$0.70</td></tr>
180
+ <tr><td>earlier demos + cancelled launches</td><td>β€”</td><td>β€”</td><td>β€”</td><td>$1.94</td></tr>
181
+ <tr><td><b>total</b></td><td></td><td></td><td></td><td><b>$2.85 / $4.70</b></td></tr>
182
+ </table>
183
+ <p class="mlabel">Append-only ledger at <code>output/job_ledger.jsonl</code> β€” the stale live job
184
+ (a 40-min timeout that didn't take) and every killed launch are logged honestly.</p>
185
+
186
+ <h2 id="run">Run it yourself</h2>
187
  <div class="note">
188
+ <p><code>python run_sim.py --mode connectome</code> β€” walk on the connectome<br>
189
+ <code>python -m flysim.brain_sim --ref-mode connectome --drive left</code> β€” steered free flight<br>
190
+ <code>python -m flysim.arena --duration 60 --n 3</code> β€” multi-fly colony with collisions<br>
191
+ <code>python -m flysim.m3 --trials 20</code> β€” the feeding experiment<br>
192
+ <code>python -m flysim.population</code> β€” the whole loop: courtship, egg, eclosion, starvation<br>
193
+ <code>python -m flysim.jobs --live --ref-mode circuit</code> β€” watch the fly fly live (HF Job)</p>
194
+ <p>Artifacts: <a href="https://huggingface.co/datasets/rf223x/fly-sim-runs">rf223x/fly-sim-runs</a> Β·
195
+ full state report in <code>REPORT.md</code> Β· every gotcha in <code>AGENTS.md</code> Β·
196
+ ownership in <code>ATTRIBUTION.md</code></p>
197
  </div>
198
 
199
+ <h2>Interactive fly, in your browser (client-side, no server)</h2>
200
+ <iframe src="https://neuromechfly.org/wasm/viewer/viewer.html" loading="lazy" title="NeuroMechFly WASM viewer"></iframe>
201
+ <p>More: <a href="https://neuromechfly.org/wasm/viewer/viewer.html">pose viewer</a> Β·
202
+ <a href="https://neuromechfly.org/wasm/game/game.html">NeuroMechFly Live slalom game</a></p>
203
+
204
+ <h2 id="credits">Credits</h2>
205
  <ul>
206
  <li><a href="https://github.com/NeLy-EPFL/flygym">FlyGym / NeuroMechFly v2</a> (Apache-2.0), Ramdya lab, EPFL β€” please cite the NeuroMechFly v2 paper</li>
207
  <li><a href="https://male-cns.janelia.org/">MaleCNS v1.0 connectome</a> (CC-BY), FlyEM/Janelia + Cambridge + MRC-LMB + Google Research</li>
208
+ <li><a href="https://www.nature.com/articles/s41586-025-09277-4">FlyBody</a> (Nature 2025, Turaga/Ramdya labs) β€” the flight body, WPG and trained policy checkpoint</li>
209
+ <li><a href="https://github.com/google-deepmind/mujoco">MuJoCo</a> (Apache-2.0) Β· <a href="https://threejs.org/">Three.js</a> (MIT)</li>
210
  </ul>
211
+
212
+ <div class="footer">
213
+ <p>Everything on this page was produced by a machine-verified metric or a rendered artifact.
214
+ The connectome selects goals; the RL policy flies the body; the CPG executes the gait.</p>
215
+ </div>
216
  </main>
217
  </body>
218
  </html>